This content originally appeared on DEV Community 👩‍💻👨‍💻 and was authored by Tanmay Prashant Rane
Preface
Amazon SageMaker is great for us data scientists and machine learning engineers for exploring data and building models. The range of preinstalled packages is great and enough for normal scenarios. But for having new and specific versions of packages "pip" or "conda" managers can be used.
What this blog will give you?
A template to bend and flex according to your requirements and get lifecycle configurations done fast. 🚀
The Problem đź§
The sole issue is every time I turn the notebook instance off, the libraries installed manually are lost. This becomes a routine every time you start the instance. I would very much like to have my environment and libraries setup as I start instances.
Preferences 🧑‍💻
- I like to get my new environment started with a fresh python installation using conda. Here's a one-liner for that:
conda create -n <env_name> python=<python version>
conda create -n dev_env python=3.9
- Then we can activate the environment as:
conda activate <env_name>
conda activate dev_env
- After getting into the environment I like to install packages using the pip manager.
pip install <package name>
pip install pyarrow
Let's put this in a lifecycle configuration shell script, shall we! đź“ś
In the following script we will take care of following tasks:
- Creation of environment with desired version of python. In this case python 3.9
- Installation of required libraries with pip manager. In this case pandas, pyarrow, and TensorFlow.
- Make newly created environment accessible for notebooks. Please read the comments in the script.
#!/bin/bash
set -e
# use ec2-user for operations
sudo -u ec2-user -i <<'EOF'
# environment creation
conda create -n dev_env python=3.9 -y
source activate dev_env
# library installation
pip install pandas
pip install pyarrow
pip install tensorflow==2.9
# make new environment accessible
conda install ipykernel -y
source deactivate
Script placement 🪄
1. Locate lifecycle configurations in console
2. Create lifecycle configuration to run on notebook start
3. Add lifecycle configuration to notebook instance configurations
Results đź’ˇ
Kernel Selection!
See the kernel listed in the dropdown đź‘€
Couple of checks on terminal
And we have what we need!
There we have it 🪄, let me know if it helped anyone. If anyone knows a better method please comment and let me know! This is only one example of endless applications of lifecycle configurations. You can access them here: SageMaker Lifecycle Configuration scripts
This content originally appeared on DEV Community 👩‍💻👨‍💻 and was authored by Tanmay Prashant Rane
Tanmay Prashant Rane | Sciencx (2022-10-02T11:33:38+00:00) SageMaker Lifecycle Configurations for custom environments. Retrieved from https://www.scien.cx/2022/10/02/sagemaker-lifecycle-configurations-for-custom-environments/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.