macaroni

… to make macaros (aka macros) …

What is a macro?

A macro is an “action” or set of “actions” that you can use to perform repetitive, time consuming, difficult, or hard to access tasks.

What is macaroni?

Macaroni is a high performance, user friendly macro system, designed to be used on PCs, Phones, and IoT devices. Macaroni integrates w/ and utilizes open ai, carbon speech sdk, carbon vision sdk, as well as other Azure Cognitive Services. Macaroni is a modern dotnet 6.0 open source application, written in C# 10.0, for Hackathon 2022.

What is a command?

A command is macaroni’s representation of a macro, using three key concepts: conditions, triggers, and executors.

When a command’s condition is true, the command is enabled, and thus triggerable. When a command’s trigger is activated, the command executes it’s list of executors, one by one.

What is a command set?

A command set is a collection of one or more commands and related data. A command set is persisted or transfered as YAML, discovered in local storage, or downloaded from websites. A command set is typically authored w/ a custom UI or w/ a text editor in YAML, optimized for human authoring and readability.

Example command set, w/ 1 command, using phrase and hotkey triggers, and using sendKeys, speak, analyze, and clipboard executors:

keyword: copilot
commands:

- name: demo

  triggers:
  - phrase: smart snippet
  - hotkey: {ctrl-shift-win-s}

  execute:
  - speak: select area on screen
  - clipboard: # clear the clipboard
  - sendKeys: {shift-win-s} # invoke snippet
  - wait:
      clipboard:
        format: image
  - analyze: {clipboard.image}
  - clipboard:
      set: {analyze.text}

 

---

 

Commands

# form
name: command name
conditions:
- (condition1, optional)
- ...
triggers:
- (trigger1, required)
- ...
execute:
- (executor1, required)
- ...

 

Conditions

# form
conditions:
- (condition)
- ...

# example
conditions:
- app: notepad

 

Triggers

# form
triggers:
- (trigger)
- ...

# example
triggers:
- phrase: paste plain text
- hotkey: {ctrl-shift-v}

 

Executors

# form
execute:
- (executor)
- ...

# example
execute:
- sendKeys: {ctrl-c}
- sendKeys: {esc}
- complete: {clipboard.text}
- insert: {complete.text}

 

Variables

# form
variables:
  (variable)
  ...

# example
variables:
  favoriteFile: 'favorites.txt'
  contacts:
  - Rob: robch@microsoft.com
  - Nic: nicchambers@microsoft.com
  music:
    files: '{folder.music}'

 

---

REFERENCE

 

Condition reference

all	Container of conditions, all true	✅
any	Container of conditions, at least 1 is true	✅
none	Container of conditions, 0 of which are true	✅
not	Container of a single condition, must be false	✅

missing	Is property “missing” (aka empty or null)	✅
required	Is property not “missing” (aka empty or null)	✅

clipboard	Clipboard contains data and/or contains a specific value or format	⚠️
global	Checks to see if a named global state is set and/or contains a specific value.	✅
file	File exists (default) and/or contains specific text	⚠️

app	Application is in focus (default) and/or running, by title or process (default either)	✅
control	Control in focus (or parents of control in focus) match control pattern/ARIA criteria	⚠️
input	Text input control in focus (default) and/or selection (true or specific text)	⚠️

javascript	Javascript function evaluates to true	✅

 

Trigger reference

phrase	When a specific linguistic pattern is typed or spoken	✅
hotkey	When a specific key w/ or wo/ modifiers, or key sequence happens	✅

message	When a specific message is broadcast from a command execution	✅
publish	When a named function is published, then invoked from a command execution	⚠️
when	When a specific condition becomes true or false or when it is changed	✅

 

Executor reference

clipboard	Sets clipboard text (default) or other formats	✅
set	sets or clears a state by name (default)	✅

file	Reads, writes, or appends to a file on disk	✅
http	Executes an HTTP request (via GET or POST)	✅
json	Parses a json string, making typical json expressions available via resolvers	✅
log	Logs a string to the application log (for diagnostics)	✅

insert	Inserts text into the application in focus	✅
sendKeys	Sends keystrokes to the application in focus	✅
mouse	Simulates mouse movement, right/left/middle clicking, +scroll wheel	⚠️

emulate	Emulates a phrase is if the user spoke it	✅
listen	Controls if speech is “listening”, “not listening” or “listening for keyword”	✅
play	Plays a beep or short sound clip from a file	✅

alert	Display message box with text (default) and optional title and image	✅
confirm	Displays yes/no message with text (default) and optional title and image	✅
display	Displays text in a transient way (text feedback)	✅
pick	Displays a selection experience, allowing user to pick from list of items	✅
prompt	Displays a prompt to the user with a text input type experience	✅
toast	Displays text in a less transient way (platform toast)	⚠️

app	Switches to an application by process or title (default either)	✅
close	Closes an application by process or title (default either)	✅
maximize	Minimizes an application by process or title (default either)	✅
minimize	Maximized an application by process or title (default either)	✅
restore	Restores an application by process or title (default either)	✅
run	Runs an application by process or shellExecute (default tries both in that order)	✅

analyze	Analyzes image (default) specified using Vision SDK	✅
complete	Completes text (default) specified using Open AI SDK	✅
translate	Translates text from one language to another language using the Translator service	✅
recognize	Recognizes text from microphone (default) or file specified using Speech SDK	⚠️
speak	Speaks text (default) or ssml to speakers (default)	✅

async	execute contained sequence of executors in a synchronous manner but asynchronous to other executors at the same level	✅
if	execute contained sequence of executors if the condition or conditions are satisfied	✅
delay	Delays for the specified amount of time	✅
quit	Signals to the hosting application that macaroni should exit	✅
random	Randly picks from a sequence of text	✅
stop	Stops executing all executors initiated by the trigger that started them	✅
wait	Waits for a set of conditions to be true	✅
while	execute contained sequence of executors while the condition or conditions are satisfied	✅

dialog	Orchestrates a multi-turn dialog scenario, containing a sequence of when executors	✅
when	Manages a single ladder logic branch of a multi-turn dialog scenario (child of dialog executor)	✅

broadcast	Broadcasts a named message and optional value to command sets and commands that might be listening	✅
invoke	Calls a named function published elsewhere in this command set (or any command set if published global)	⚠️

javascript	Executes javascript code, w/ object model provided for the command, w/ access to host and methods equivalent to all executors	✅

android.intent	Launches an Android action based on uri or additional specified extras	✅

 

Variable reference

files		✅