This content originally appeared on DEV Community and was authored by Mfonobong Uyah
In software development, where efficiency, consistency, and reliability are paramount, automation tools play a crucial role. Makim, an innovative open-source tool, steps into the spotlight to improve automation workflows. It simplifies script execution, environment management, and task dependencies, positioning itself as a great asset in modern development environments.
Introducing Makim
Makim elevates project automation by offering a structured, yet flexible approach to manage routine tasks, complex task dependencies, and environment configurations. Its design is centered around the .makim.yaml configuration file, allowing developers to orchestrate their workflows with precision and ease. Unlike traditional script execution tools, Makim's Python-based architecture and support for multiple programming languages and shells enhance its versatility and applicability across diverse projects.
Especially suited for DevOps Engineers and Software Developers, Makim eliminates redundancy in automation tasks. Its core functionality extends beyond simple script execution, encompassing:
Argument definition for scripts
Organization of tasks into groups
Advanced dependency management between tasks
Utilization of environment variables and custom variables
Dynamic content generation with Jinja2 templates
Specification of working directories for tasks
Execution flexibility through support for multiple interpreters or shells
Despite its broad capabilities, Makim does not directly support Windows but it can be used on a Windows machine via WSL.
Getting Started with Makim
Installation
Makim can be installed via pip or conda, catering to different setup preferences:
- To install Makim using pip, run:
!pip install -q "makim==1.15.2"
- For those who prefer conda, execute:
conda install "makim=1.15.2”
Given Makim's active development, pinning to a specific version is recommended to ensure consistency.
For this tutorial, we will disable the output color feature provided by typer, the command-line interface engine used by Makim.
import os
os.environ["NO_COLOR"] = "1"
Configuring .makim.yaml
The .makim.yaml file is the foundation of your Makim configuration. Here's how to start:
Create the .makim.yaml File: Place this file at the root of your project directory.
Define Your Automation Tasks: Configure your tasks, specifying actions, arguments, and dependencies. For example:
%%writefile .makim.yaml
version: 1.0.0
groups:
clean:
env-file: .env
tasks:
tmp:
help: Use this task to clean up temporary files
run: |
echo "Cleaning up..."
tests:
tasks:
unit:
help: Build the program
args:
clean:
type: bool
action: store_true
help: if not set, the clean dependency will not be triggered.
dependencies:
- task: clean.tmp
if: ${{ args.clean == true }}
run: |
echo "Running unit tests..."
OUTPUT
Overwriting .makim.yaml
This setup demonstrates Makim's ability to manage tasks with conditional logic and dependencies.
Exploring Makim's CLI
Makim's CLI provides insights into available commands, arguments, and configurations through the auto-generated help menu:
!makim --help
OUTPUT
Usage: makim [OPTIONS] COMMAND [ARGS]...
Makim is a tool that helps you to organize and simplify your helper commands.
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --version -v Show the version and exit │
│ --file TEXT Makim config file [default: .makim.yaml] │
│ --dry-run Execute the command in dry mode │
│ --verbose Execute the command in verbose mode │
│ --install-completion Install completion for the current │
│ shell. │
│ --show-completion Show completion for the current shell, │
│ to copy it or customize the │
│ installation. │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ clean.tmp Use this task to clean up temporary files │
│ tests.unit Build the program │
╰──────────────────────────────────────────────────────────────────────────────╯
If you have any problem, open an issue at:
https://github.com/osl-incubator/makim
This feature facilitates easy access to Makim's functionalities, enhancing usability and understanding of the tool.
Executing Your First Commands
With your .makim.yaml file set up, you can begin to use makim:
!makim clean.tmp
OUTPUT
Makim file: .makim.yaml
Cleaning up...
!makim tests.unit
OUTPUT
Makim file: .makim.yaml
Running unit tests...
If you happen to type your command wrongly, Makim will suggest an alternative:
!makim tests.unittest
OUTPUT
Usage: makim [OPTIONS] COMMAND [ARGS]...
Try 'makim --help' for help.
╭─ Error ──────────────────────────────────────────────────────────────────────╮
│ No such command 'tests.unittest'. │
╰──────────────────────────────────────────────────────────────────────────────╯
Command tests.unittest not found. Did you mean tests.unit'?
Makim CLI is empowered by Typer, and it allows us to have auto-completion for Makim groups and tasks! If you want to install, you can run the following command:
!makim --install-completion
OUTPUT
bash completion installed in /home/xmn/.bash_completions/makim.sh
Completion will take effect once you restart the terminal
After this command you will need to restart the terminal in order to use this auto-completion feature.
Advanced Features and Examples
Makim's adaptability is showcased through various features and practical examples:
- Conditional Dependencies and Arguments: Define complex task dependencies with conditional execution based on passed arguments.
- Dynamic Configuration with Jinja2: Leverage Jinja2 templates for advanced scripting and dynamic content generation.
- Environment and Custom Variable Management: Organize and utilize variables effectively across different scopes of your project.
- Specifying Working Directories: Control the execution context of your tasks by setting working directories. These examples underscore Makim's capability to accommodate intricate automation scenarios, streamlining development workflows.
Exploring Makim Through Examples
Utilizing Various Interpreters
Makim extends its functionality beyond conventional script execution by supporting various interpreters and shell languages, facilitating a versatile development environment. While xonsh is the default interpreter - blending the capabilities of Bash and Python for an enriched command-line experience - Makim's architecture allows for seamless integration with other environments. For developers seeking to leverage this feature, a foundational understanding of xonsh can be beneficial. Comprehensive details and usage guidelines are available in the official xonsh documentation.
This section demonstrates executing straightforward commands across multiple interpreters, showcasing Makim's adaptability to diverse programming contexts.
%%writefile .env
MSG_PREFIX="Running Makim: Hello, World,"
OUTPUT
Overwriting .env
%%writefile .makim.yaml
version: 1.0
env-file: .env
groups:
tests:
tasks:
node:
help: Test using nodejs
shell: node
run: console.log("${{ env.MSG_PREFIX }} from NodeJS!");
perl:
help: Test using perl
shell: perl
run: print "${{ env.MSG_PREFIX }} from Perl!\n";
python:
help: Test using php
shell: python
run: print("${{ env.MSG_PREFIX }} from Python!")
r:
help: Test using R
shell: Rscript
run: print("${{ env.MSG_PREFIX }} from R!")
sh:
help: Test using sh
shell: sh
run: echo "${{ env.MSG_PREFIX }} from sh!"
run-all:
help: Run tests for all the other tasks
dependencies:
- task: node
- task: perl
- task: python
- task: r
- task: sh
OUTPUT
Overwriting .makim.yaml
!makim --help
OUTPUT
Usage: makim [OPTIONS] COMMAND [ARGS]...
Makim is a tool that helps you to organize and simplify your helper commands.
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --version -v Show the version and exit │
│ --file TEXT Makim config file [default: .makim.yaml] │
│ --dry-run Execute the command in dry mode │
│ --verbose Execute the command in verbose mode │
│ --install-completion Install completion for the current │
│ shell. │
│ --show-completion Show completion for the current shell, │
│ to copy it or customize the │
│ installation. │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ tests.node Test using nodejs │
│ tests.perl Test using perl │
│ tests.python Test using php │
│ tests.r Test using R │
│ tests.run-all Run tests for all the other tasks │
│ tests.sh Test using sh │
╰──────────────────────────────────────────────────────────────────────────────╯
If you have any problem, open an issue at:
https://github.com/osl-incubator/makim
Prior to executing these tasks, it is necessary to install the required dependencies:
!mamba install -q -y perl nodejs r-base sh
Proceed to execute all defined tasks by invoking the run-all task, which encapsulates all other tasks as its dependencies for a sequential execution process:
!makim tests.run-all
OUTPUT
Makim file: .makim.yaml
Running Makim: Hello, World, from NodeJS!
(node:1634785) Warning: The 'NO_COLOR' env is ignored due to the 'FORCE_COLOR' env being set.
(Use `node --trace-warnings ...` to show where the warning was created)
Running Makim: Hello, World, from Perl!
Running Makim: Hello, World, from Python!
[1] "Running Makim: Hello, World, from R!"
Running Makim: Hello, World, from sh!
In scenarios where your chosen interpreter supports debugging - such as Python or Xonsh through the use of breakpoint() - you can introduce a breakpoint within your code. This enables the debugging of your Makim task, allowing for an interactive examination of the execution flow and variable states.
Using Variables (vars)
Makim facilitates the definition of variables within the .makim.yaml configuration, supporting all the YAML data types, including strings, lists, and dictionaries. This feature enhances script configurability and reusability across different tasks and environments.
Consider reviewing the provided example to understand how to effectively leverage variables in your Makim configurations:
%%writefile .makim.yaml
version: 1.0
vars:
project-name: "my-project"
dependencies:
"dep1": "v1"
"dep2": "v1.1"
"dep3": "v2.3"
authorized-users:
- admin1
- admin2
- admin3
groups:
staging:
vars:
env-name: "staging"
staging-dependencies:
"dep4": "v4.3"
"dep5": "v1.1.1"
staging-authorized-users:
- staging1
- staging2
- staging3
tasks:
create-users:
help: Create users for staging, this example uses jinja2 for loop.
# each task can also specify their `vars`, but it will not be used in this example
run: |
def create_user(username):
print(f">>> creating user: {username} ... DONE!")
print("create admin users:")
{% for user in vars.authorized_users %}
create_user("${{ user }}")
{% endfor %}
print("\ncreate staging users:")
{% for user in vars.staging_authorized_users %}
create_user("${{ user }}")
{% endfor %}
install:
help: install deps for staging using native xonsh `for` loop (it could work with Python as well)
# each task can also specify their `vars`, but it will not be used in this example
run: |
def install(package, version):
print(f">>> installing: {package}@{version} ... DONE!")
print("install global dependencies:")
for package, version in ${{ vars.dependencies | safe }}.items():
install(package, version)
print("\ninstall staging dependencies:")
for package, version in ${{ vars.staging_dependencies | safe }}.items():
install(package, version)
OUTPUT
Overwriting .makim.yaml
Now, let's proceed to create users within the staging environment:
!makim staging.create-users
OUTPUT
Makim file: .makim.yaml
create admin users:
>>> creating user: admin1 ... DONE!
>>> creating user: admin2 ... DONE!
>>> creating user: admin3 ... DONE!
create staging users:
>>> creating user: staging1 ... DONE!
>>> creating user: staging2 ... DONE!
>>> creating user: staging3 ... DONE!
Last but not least, let's run the install task:
!makim staging.install
OUTPUT
Makim file: .makim.yaml
install global dependencies:
>>> installing: dep1@v1 ... DONE!
>>> installing: dep2@v1.1 ... DONE!
>>> installing: dep3@v2.3 ... DONE!
install staging dependencies:
>>> installing: dep4@v4.3 ... DONE!
>>> installing: dep5@v1.1.1 ... DONE!
Defining Arguments
Makim enhances script flexibility by allowing the use of arguments. It enables not only the definition of arguments for tasks but also the passing of arguments to dependencies and the specification of conditions for those dependencies.
Explore this functionality through this example:
%%writefile .makim.yaml
version: 1.0.0
groups:
print:
env-file: .env
tasks:
name:
help: Print given name
args:
name:
type: str
required: true
run: print("${{ args.name }}")
list:
help: Build the program
args:
i-am-sure:
type: bool
dependencies:
- task: print.name
if: ${{ args.i_am_sure == true }}
args:
name: Mary
- task: print.name
if: ${{ args.i_am_sure == true }}
args:
name: John
- task: print.name
if: ${{ args.i_am_sure == true }}
args:
name: Ellen
- task: print.name
if: ${{ args.i_am_sure == true }}
args:
name: Marc
OUTPUT
Overwriting .makim.yaml
!makim print.list
OUTPUT
Makim file: .makim.yaml
!makim print.list --i-am-sure
OUTPUT
Makim file: .makim.yaml
Mary
John
Ellen
Marc
Utilizing Environment Variables
The previous sections demonstrated the use of environment variables. Here, we'll delve into their application in more detail.
Makim permits the incorporation of environment variables from .env files or directly within the .makim.yaml file, applicable at global, group, and task levels.
Examine an example to understand the implementation:
%%writefile .env
ENV=dev
OUTPUT
Overwriting .env
%%writefile .makim.yaml
version: 1.0
env-file: .env
env:
GLOBAL_VAR: "1"
groups:
global-scope:
env:
GROUP_VAR: "2"
tasks:
test-var-env-file:
help: Test env variable defined in the global scope from env-file
run: |
import os
assert str(os.getenv("ENV")) == "dev"
test-var-env:
help: Test env variable defined in the global scope in `env` section
env:
TASK_VAR: "3"
run: |
import os
# you can get an environment variable directly with xonsh/python
assert str(os.getenv("GLOBAL_VAR")) == "1"
# or you can get an environment variable using jinja2 tag
assert "${{ env.GROUP_VAR }}" == "2"
assert "${{ env.get("TASK_VAR") }}" == "3"
assert "${{ env.get("UNKNOWN_TASK_VAR", "4") }}" == "4"
OUTPUT
Overwriting .makim.yaml
!makim global-scope.test-var-env-file
OUTPUT
Makim file: .makim.yaml
!makim global-scope.test-var-env
OUTPUT
Makim file: .makim.yaml
Specifying the Working Directory
Makim provides the capability to set a specific working directory for tasks at any scope: global, group, or task.
Review a straightforward example to learn how to apply this feature:
`%%writefile .makim.yaml
version: 1.0
working-directory: "/tmp"
groups:
check-wd:
tasks:
is-tmp:
help: Test if working directory is `tmp`
run: |
import os
print(os.getcwd())
assert os.getcwd() == "/tmp"
OUTPUT
Overwriting .makim.yaml
!makim check-wd.is-tmp
OUTPUT
Makim file: .makim.yaml
/tmp
This tutorial concludes with a showcase of Makim's key features. While this overview covers the essentials, diving deeper into Makim will reveal more advanced and intriguing ways to leverage its capabilities.
Contributing to Makim
Makim's growth is propelled by its community. Contributions, whether through code, documentation, or feedback, are welcome. Explore the GitHub repository and consider contributing to foster Makim's development.
Tools Similar to Makim
Make:
Make decides which part of a large scale program needs to be recompiled. It is often used in C and C++ languages.
Taskfile:
Taskfile is described as a task runner/build tool that aims to be simpler and easier to use. It is often used in the Go language.
Nox:
Nox is a command line tool that automates testing in multiple Python environments. It uses a standard Python file for configuration.
Ansible:
Ansible is an IT automation engine that automates provisioning, configuration, management, app deployment, orchestration, and many other IT processes.
Conclusion
Makim stands out as a transformative tool in project automation, bridging the gap between simplicity and complexity. Its comprehensive feature set, coupled with the flexibility of its configuration, makes Makim a quintessential tool for developers and DevOps engineers alike. As you incorporate Makim into your projects, its impact on enhancing productivity and consistency will become evident, marking it as an indispensable part of your development toolkit.
Dive deeper into Makim's functionalities by visiting the official documentation. Try it and let us know your thoughts about it!
This content originally appeared on DEV Community and was authored by Mfonobong Uyah
Mfonobong Uyah | Sciencx (2024-07-20T03:23:41+00:00) Streamlining Project Automation with Makim. Retrieved from https://www.scien.cx/2024/07/20/streamlining-project-automation-with-makim/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.