luau: recipe support

[jqnatividad]

As qsv's domain-specific language (DSL), luau can be used to create recipes - short reusable, task-oriented, composable, data-wrangling scripts that can combine several commands (and not just qsv commands, any other CLI command or module that can be called from luau).

Example tasks include:

• geocoding using your geocoder of choice
• screening for PII
• normalizing postal addresses
• enriching data with high-value datasets (e.g. census) and other datasets - from their own catalog, or from catalogs of other users
• auto-classify/tag datasets
• etc.

We consciously used the word "recipe" to connote that these scripts are targeted more to data analysts, rather than developers.

It also drove us to adopt luau as DSL in our aim to democratize data-wrangling. Much the same way Roblox chose Luau to let "non-developer" gamers to program games and play games created by other "non-developer" gamers - we want qsv users to create recipes and use recipes created by other users.

To fully realize this vision/mission, luau will need to be extended to have more robust recipe support:

• Similar to the BEGIN and END block language extensions we introduced for qsv luau, we need to a RECIPE block at the beginning of a proper qsv recipe.
• The RECIPE block will have:
  • Description of the recipe
  • Ingredients. There are two kinds of ingredients:
    • Global - are immutable ingredients/variables that are available while the recipe is running. Globals , as the name implies, can be accessed globally at any stage of a recipe's execution (i.e. BEGIN, MAIN and END blocks).
    • Column - are the column names of the ingredients that are retrieved for EACH row of the CSV. These are then mapped and available in the MAIN LOOP BLOCK of a recipe as it loops through a CSV.