Now that we understand how to keep an eye on the data going _into_ the model, Part 2 focuses on what comes _out_ and how to manage the monitoring process effectively. We'll dive into monitoring model predictions and actual performance, automating these checks with Apache Airflow, visualizing the results for clear insights, and establishing strategies for responding when drift inevitably occurs.

Continuous Model Monitoring in Machine Learning (Part 2): Performance, Automation, and Action

Continuous Model Monitoring in Machine Learning (Part 1)

Continuous Model Monitoring in Machine Learning (Part 1): Why It's Crucial and How to Monitor Your Inputs

# 1. Introduction
This blog discusses how to effectively interact with popular BI tools by setting up data models for these tools. These
BI systems typically rely on structured data extracted from the underlying source system databases. Therefore,
structuring the data model before diving into metric calculations and building interactive dashboards with filters
efficiently remains a challenge for Data Analysts.
# 2. Principal steps
### 2.1 Definition and purpose of data modeling in the context of BI:
In the landscape of Business Intelligence (BI), data modeling stands as a fundamental pillar, essential for harnessing
the full potential of organizational data and transforming it into actionable insights.
**Data model** is:

— It is the blueprint for designing a database
— They visual representations of an enterprise’s data elements and the connections between them.
— visually represents the nature of the data, business rules governing the data, and how the data will be organized in a
database.
— Defines and structure data in the context of relevant business processes, data models support the development of
effective information systems.

### 2.2 Normalization and denormalization database
**Definitions**
- Normalization: According to Wikipedia’s definitions, **Database normalization** is the process of structuring a
[relational database](https://en.wikipedia.org/wiki/Relational_database "Relational database") in accordance with a
series of so-called _**[normal forms](https://en.wikipedia.org/wiki/Database_normalization#Normal_forms)**_ in order to
reduce [data redundancy](https://en.wikipedia.org/wiki/Data_redundancy "Data redundancy") and improve [data
integrity](https://en.wikipedia.org/wiki/Data_integrity "Data integrity").
- As the opposite, **Denormalization** is the inverse process of normalization where the redundancy is added to the data
intentionally to improve the performance of the specific application and data integrity.
**Application** in designing schema:
- **Normalized**: A schema design to store non-redundant and consistent data
- Data integrity is maintained
- Little to no redundant data
- Many tables
- Optimizes for storage of data
![1.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/b5d7050867f7c4c91b47e725aea57da9a093715ceaae657d0199a4f27c8641f9.jpg)
- **Denormalized**: A schema that combines data so that accessing data (querying) is fast.
- Data integrity is not maintained
- Redundant data is common
- Fewer tables
- Excessive data, storage is less optimal.

![2.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/c6a3b2b95a623feeed489bd977554994f2b406123441425b3d0b7cf96e24082c.jpg)

**Popular tools:**
Not all BI tools are created equal. Some are designed for specific purposes, such as data visualization, reporting, or
analytics, while others offer a more comprehensive suite of features and functions.
Depending on the strategy and data usage orientation of companies, they typically use the following popular BI tools:
- Microsoft Power BI (when the company primarily uses Microsoft services)
- Tableau
- Looker Studio (when the company primarily uses Google services)
- Excel
Typically, for BI tools with strong support for modeling, such as Power BI and Tableau, you can use a normalized
database. However, for BI tools that do not support data modeling (Looker Studio, Excel,....), you should use a
denormalized database to ensure all metrics and dimensions are fully covered.
In the following sections, I will delve into data model design for BI tools such as Power BI and Tableau.
### 2.3 Establish Relationships
Relationships define how tables are connected and interact with each other. They are crucial for maintaining data
integrity and ensuring accurate query results.
Each entity (table) can potentially have a relationship with every other one, but those relationships are typically one
of three types:
- **One-to-one relationships**:
- A _one-to-one relationship_ (1:1) is a relationship where a record in one table is associated with exactly one record
in another table.
- Primary Key as Foreign Key

![3.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/51b98bf0ef2d4fc7f36b539e137d2771702352d119dcafb0bfa43db15cf39fe4.jpg)

Unless you have a good reason not to, a 1:1 relationship usually indicates that you’d be better off combining the two
tables’ data into a single table.
However, you might want to create tables with a 1:1 relationship under a particular set of circumstances. If you have a
field with optional data, such as “description,” that is blank for many of the records, you can move all of the
descriptions into their own table, eliminating empty space and improving database performance. Or if you have sensitive
data that needs to be protected for customers and don't want everyone to be able to see it, you can store it in a
separate table, grant permissions, and create a 1:1 relationship with the original Customer table.
To guarantee that the data matches up correctly, you’d then have to include at least one identical column in each table,
most likely the primary key.
- **One-to-many relationships (highly recommended)**:
- One row in a table (usually dimension table) can have a relationship with many rows in another table (usually fact
table).
- To implement a 1:M relationship as you set up a database, simply add the primary key from the “one” side of the
relationship as an attribute in the other table. When a primary key is listed in another table in this manner, it’s
called a foreign key.

![4.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/bb50e75439cf15a6d7aae3924a55fbce649fec6e310af2352c30c614fd0ff232.jpg)

For example, a single category might have placed many product at once.
- **Many-to-many relationships (not recommended)**:
- When multiple entities from a table can be associated with multiple entities in another table, they are said to have a
many-to-many (M:N) relationship. This might happen in the case of students and classes, since a student can take many
classes and a class can have many students.

![5.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/ee7b8724163acd4858b91742086c09d87f0617aca45fde93ef8e0fa1792addcb.jpg)

- Unfortunately, it’s not directly possible to implement this kind of relationship in a database. Instead, you have to
break it up into two one-to-many relationships.
- To do so, create a new entity between those two tables.

### 2.4 Database schema types
Typically, the data models built for BI tools are mostly dimensional modeling, which extends logical and physical data
models to further model data and data relationship requirements. Database schemas that are modeled according to
dimensional modeling principles work well with applications that must read large amounts of data quickly.
- *Star schema:* is a type of relational database schema that is composed of a single, central fact table that is
surrounded by dimension tables. A star schema can have any number of dimension tables. The branches at the end of the
links connecting the tables indicate a many-to-one relationship between the fact table and each dimension table.

![7.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/d3a51fd0a8edc12b5c795ad9ef65214758d6bc27497bff4b56d46bfe8ed9ad41.jpg)
(*Model built by Power BI*)

- *Snowflake schemas*: consists of one fact table that is connected to many dimension tables, which can be connected to
other dimension tables through a many-to-one relationship. Tables in a snowflake schema are usually normalized to the
third normal form. Each dimension table represents exactly one level in a hierarchy.
![8.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/dfbad8e2d94a0e37c94d0328c76e1874c05d4897379e049cb55a6611e7392b1e.jpg)
(*Model built by Power BI*)

***Choose the appropriate data modeling technique***:
| Star Schema | Snowflake Schema |
| ----------------------------- | ----------------------------------------------- |
| Top-down model | Bottom-up model |
| Less time for query execution | More time for query execution |
| Occupies more space | Uses less space |
| Denormalized is used | Both normalization and denormalization are used |
| Low query complexity | Higher query complexity |
| Less number of foreign keys | More number of foreign keys |
| High data redundancy | Low data redundancy |
| Simple design | Complex design |
| Easy to understand | Hard to understand |
*Key differences table between star schema and snowflake. | Image: Dilip Kumar Patairya*
### 2.6 Optimize Performance
- Index key column (primary key): Columns with high cardinality, such as unique identifiers
- Partition to manage and access subsets of data more effectively
- Materialize to store query results for frequent use.
- Store data in the correct format and character encoding. The default format of the database often allows up to 1000
characters; however, for fields like product_name or merchant_name, 1000 characters are unnecessary. Keeping the default
setting consumes a lot of resources, which can slow down reporting.
- Select only the necessary columns from a table for reporting.
### 2.7 Validate Model
- Ensure that the model meets all business requirements and performs as expected.
- Test it with real queries and evaluating whether it returns accurate and timely results.
### 3. Conclusion
The data modeling lifecycle in BI projects is a structured and iterative process that encompasses requirement analysis,
conceptual modeling, logical modeling, physical modeling, implementation, testing, maintenance, and evolution. A
well-structured data model lays the foundation for a BI solution by defining how data is connected, stored, and
accessed. Follow the key steps above, to effectively construct a data model tailored for a BI solution, empowering you
to unlock the full potential of your data.


### References:
- https://www.lucidchart.com/pages/database-diagram/database-design
- https://medium.com/@santosh_beora/database-schemas-star-schema-vs-snowflake-schema-528163c4215d
- https://medium.com/@santosh_beora/dimension-and-fact-tables-b88283c96e0b
- https://medium.com/@gadnwachukwu2/the-basics-of-data-modeling-in-business-intelligence-652254020d0d
- https://builtin.com/articles/star-schema-vs-snowflake-schema#:~:text=If%20you%20have%20a%20few,will%20be%20a%20better%20scheme.
- https://www.lucidchart.com/pages/database-diagram/database-design

his blog discusses how to effectively interact with popular BI tools by setting up data models for these tools. These BI systems typically rely on structured data extracted from the underlying source system databases. Therefore, structuring the data model before diving into metric calculations and building interactive dashboards with filters efficiently remains a challenge for Data Analysts.

How to build effective model for BI tool

# Versioning Data in MLOps with DVC (Data Version Control)
## 1. Introduction
Have you ever worked on a machine learning project and found yourself drowning in data files? Maybe you've accidentally overwritten important data, or **you can't remember which dataset version gave you the best results**. 
![1.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/fcfa0847e776c5919fd4e9b517c6fdbac2be480c4b750e6a5c9abf2350e169fe.jpg)

Enter DVC, or Data Version Control. It's a tool that helps **manage and version your data**, much like how Git manages your code. 

## 2. Understanding Data Versioning in MLOps
Data versioning is about tracking changes in datasets over time. It's essential in machine learning operations (MLOps) for:
- Reproducing results
- Facilitating collaboration
- Tracking data lineage
  
![2.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/0926624668c6b98817b8d26dd22cc1303a54842b5128632b6814c3f9d125f699.jpg)

## 3. Introduction to DVC
DVC is an open-source tool that works alongside Git to handle large files, datasets, and machine learning models. Key features:
- Tracks changes in data without storing the data itself in Git
- Works with existing Git workflows
- Connects with cloud storage for large files
![3.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/2165695c1812fa81775292080190935a9a556a689bcc41eb7ad7710c27136d6b.jpg)
## 4. Setting up DVC
Let's set up a project with DVC:
1. Create a new directory and initialize Git:
  ```bash
mkdir my_ml_project
cd my_ml_project
git init
   ```
2. Install DVC:
  ```bash
pip install dvc==3.55.2
   ```
3. Initialize DVC:
  ```bash
dvc init
git commit -m "Initialize DVC"
   ```
Your project structure now looks like this:
```
📦my_ml_project  
 ┣ 📂.dvc  
 ┃ ┣ 📂tmp  
 ┃ ┃ ┗ 📜btime  
 ┃ ┣ 📜.gitignore  
 ┃ ┗ 📜config  
 ┣ 📂.git  
 ┗ 📜.dvcignore
```
![4.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/c067b95b990083839c495592b5cacbd00fcc6a2486b5b8a2c6adf09408c62078.jpg)
## 5. Basic DVC Workflow
Let's walk through a basic DVC workflow using an image classification project as an example.
1. Set up your project structure:
   ```
   my_ml_project/
   ├── data/
   │   └── images/
   ├── model.py
   ├── train.py
   └── .gitignore
   ```
   
   This structure keeps your project organized. The `data` directory will store your datasets, while separate Python files will handle model definition and training.
2. Add your dataset to DVC:
   ```bash
   dvc add data/images
   ```
   This command tells DVC to start tracking the `images` directory. DVC creates a small metadata file (`data/images.dvc`) that Git will version, while the actual image data is stored separately. This approach allows you to version your data without bloating your Git repository.
   The `data/images.dvc` file will look something like this:
   ```yaml
      outs:
      - md5: d41d8cd98f00b204e9800998ecf8427e  # Unique identifier for the current state of the data
        size: 0  # Total size of the tracked data
        path: images  # Location of the data relative to this .dvc file
      ```
3. Commit changes to Git:
   ```bash
   git add data/images.dvc .gitignore
   git commit -m "Add image dataset"
   ```
   By committing the DVC metadata file, you're essentially versioning a pointer to your data. This allows you to track which version of the data was used at this point in your project's history.
4. Create a simple training script (`train.py`):
   ```python
   import numpy as np
   from sklearn.model_selection import train_test_split
   from sklearn.ensemble import RandomForestClassifier
   from sklearn.metrics import accuracy_score
   import joblib
   import json
   def load_image_data(path):
          # Simulating image data loading
          # In a real scenario, you'd load and preprocess images here
          X = np.random.rand(1000, 224, 224, 3)  # 1000 images of 224x224 with 3 color channels
          y = np.random.randint(0, 10, 1000)  # 10 classes
          return X, y
   # Load and preprocess data
      X, y = load_image_data("data/images/")
      X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
   # Train model
      model = RandomForestClassifier(n_estimators=100)
      model.fit(X_train.reshape(X_train.shape[0], -1), y_train)
   # Evaluate model
      y_pred = model.predict(X_test.reshape(X_test.shape[0], -1))
      accuracy = accuracy_score(y_test, y_pred)
      print(f"Model accuracy: {accuracy}")
   # Save model
      joblib.dump(model, "model.joblib")
   # Save accuracy for DVC to track
      with open("accuracy.json", "w") as f:
          json.dump({"accuracy": accuracy}, f)
      ```
   This script defines our machine learning pipeline. It loads data, trains a model, evaluates it, and saves both the model and its accuracy. In a real scenario, you'd replace the data loading function with actual image processing code.
5. Create a DVC pipeline (`dvc.yaml`):
   ```yaml
   stages:
     train:
       cmd: python train.py  # Command to run for this stage
       deps:
         - data/images  # Input data dependency
         - train.py  # Code file dependency
       outs:
         - model.joblib  # Output model file
       metrics:
         - accuracy.json:  # Output metric file
             cache: false
   ```
   This YAML file defines a DVC pipeline. It specifies how to run your training script, what it depends on, and what it produces. DVC uses this information to determine when the pipeline needs to be re-run based on changes to the inputs, ensuring your outputs are always up-to-date.
6. Run the DVC pipeline:
   ```bash
   dvc repro
   ```
   This command runs the pipeline defined in `dvc.yaml`. DVC checks if any dependencies have changed and, if so, re-runs the affected stages. This ensures your outputs are always consistent with your inputs and code.
7. Commit changes:
   ```bash
   git add .
   git commit -m "Add training pipeline"
   ```
   This step versions your pipeline definition, training script, and the current state of your model (via the DVC metadata files). It captures the entire state of your ML project at this point, making it easy to reproduce or revert to this exact state later.

By following these steps, you've created a versioned, reproducible machine learning pipeline. You can now track changes to your data, code, and model performance over time, and easily revert to previous states if needed. 
## 6. Advanced DVC Features
DVC offers advanced features for more complex workflows:
### Data Pipelines
DVC pipelines allow you to define and run multi-stage data processing and model training workflows. This is particularly useful for more complex projects where you have multiple interdependent steps.
Here's an example `dvc.yaml` for a multi-stage pipeline:
```yaml
stages:
  preprocess:
    cmd: python preprocess.py  # Command to run the preprocessing script
    deps:
      - data/raw  # Input raw data
    outs:
      - data/processed  # Output processed data
  train:
      cmd: python train.py  # Command to run the training script
      deps:
        - data/processed  # Input processed data
      outs:
        - model.joblib  # Output trained model
      metrics:
        - accuracy.json:  # Output accuracy metric
            cache: false
  evaluate:
      cmd: python evaluate.py  # Command to run the evaluation script
      deps:
        - model.joblib  # Input trained model
        - data/test  # Input test data
      metrics:
        - evaluation.json:  # Output evaluation metrics
            cache: false
  ```
This pipeline defines three stages: preprocessing, training, and evaluation. Each stage has its own dependencies and outputs, allowing DVC to determine which stages need to be re-run when something changes.
### Metrics Tracking
DVC can track and compare metrics across experiments, which is crucial for monitoring your model's performance over time.
To view metrics:
```bash
dvc metrics show  # Displays the current metrics
```
To compare metrics with a previous version:
```bash
dvc metrics diff  # Shows how metrics have changed since the last commit
```
These commands help you quickly assess how changes in your data or code have affected your model's performance.
### Remote Storage
DVC can store your data in remote locations like S3, Google Cloud Storage, or SSH servers. This is essential for backing up your data and collaborating with team members.
To add a remote storage:
```bash
dvc remote add -d myremote s3://mybucket/dvcstore
# -d sets this remote as the default
# myremote is the name we're giving to this remote
# s3://mybucket/dvcstore is the S3 bucket path
```
To push data to the remote:
```bash
dvc push  # Uploads tracked data to the remote storage
```
To pull data from the remote:
```bash
dvc pull  # Downloads tracked data from the remote storage
```
Using remote storage allows you to keep your Git repository light while still having full version control over your large datasets and models.
### Experiment Management
DVC offers features to help manage and compare different experiments:
```bash
dvc exp run --set-param train.epochs=10  # Run an experiment with a specific parameter
dvc exp show  # Display a table of experiments and their metrics
dvc exp apply <exp-id>  # Apply the changes from a specific experiment
```
These commands allow you to easily try different parameters, compare results, and apply the most promising changes to your project.

## 7. Best Practices
1. Use meaningful stage names in your DVC pipeline.
2. Keep your raw data separate from processed data.
3. Use `.gitignore` to prevent large files from being added to Git.
4. Regularly push your data to remote storage for backup and collaboration.
5. Use metrics to track and compare experiment results.
## 8. Conclusion
DVC brings the power of version control to your data and ML pipelines. By using DVC, you can:
- Track changes in your datasets
- Create reproducible workflows
- Collaborate more effectively with your team
- Manage experiments and compare results easily

Enter DVC, or Data Version Control. It's a tool that helps **manage and version your data**, much like how Git manages your code. 

Versioning Data in MLOps with DVC (Data Version Control)

# Rate Limiter in Triton: Managing Resource Allocation for MLOps
## 1. Introduction
Deploying machine learning models can be tricky. One big challenge is managing computer resources like memory and processing power (GPU utilization). 
When you're running multiple models at once::
- Your GPU VRam might run out of memory
- Some models might use up all the resources, leaving others waiting
- The whole system could slow down or crash
This is where Triton Inference Server comes in handy. Triton is a tool that helps deploy and run machine learning models. It has a feature called the Rate Limiter, which helps manage resources better.
In this tutorial, we'll learn about Triton and its Rate Limiter. We'll start with the basics and then see how the Rate Limiter can solve resource problems.
## 2. Triton Inference Server Fundamentals
### What is Triton Inference Server?
Triton Inference Server is like a helpful manager for your machine learning models. It's **a software** that:
- Runs your models
- Handles requests to use these models
- Manages resources for smooth operation
### Key Features of Triton
- Supports many types of models (TensorFlow, PyTorch, ONNX, etc.)
- Can run multiple models at the same time
- Allows models to run on CPUs or GPUs
- Has tools to manage resources (like the Rate Limiter we'll learn about)
### Basic Architecture
Let's imagine Triton as a building with many rooms:
- Each room can hold a different model
- There's a front desk (API) where requests come in
- The building manager (Triton) decides **which** room (model) to use and **when**
## 3. Getting Started with Triton
Let's set up a simple Triton server and run a basic model.
### Installation
1. The easiest way to get started is using Docker. Install Docker if you haven't already.
2. Pull the Triton Docker image:
```shell
docker pull nvcr.io/nvidia/tritonserver:23.09-py3-sdk
```
### Running a Simple Triton Server
1. Create a folder for your models:
   ```shell
   mkdir -p models/simple_model/1
   ```
2. For this example, let's use a pre-trained ONNX model. Download a simple model (like a small image classifier) and place it in `models/simple_model/1/model.onnx`.
3. Create a configuration file `models/simple_model/config.pbtxt`:
   ```txt
   name: "simple_model"
   platform: "onnxruntime_onnx"
   max_batch_size: 8
   input [
     {
       name: "input"
       data_type: TYPE_FP32
       dims: [ 3, 224, 224 ]
     }
   ]
   output [
     {
       name: "output"
       data_type: TYPE_FP32
       dims: [ 1000 ]
     }
   ]
   ```
4. Start the Triton server:
```shell
docker run --gpus all --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v $(pwd)/models:/models nvcr.io/nvidia/tritonserver:21.09-py3 tritonserver --model-repository=/models
```
You now have a Triton server running with a simple model!

**Note:**
- The `--gpus all` flag ensures Triton can access all available GPUs.
- Use `nvidia-smi` to verify GPU availability before starting Triton.

## 4. Understanding the Rate Limiter
### What is a Rate Limiter?
A Rate Limiter is like a traffic controller for your models. It decides:
- How many requests each model can handle at once
- Which models get to run when resources are limited
### How Triton's Rate Limiter Works
Triton's Rate Limiter uses two main concepts:
1. Resources: These are things your models need, like GPU memory or CPU power.
2. Priorities: This decides which models are more important when resources are limited.
The Rate Limiter keeps track of available resources and assigns them to models based on their needs and priorities.

## 5. Configuring the Rate Limiter in Triton
Let's set up the Rate Limiter for our Triton server.
### Defining Resources
Resources are identified by a unique name and a count indicating the number of copies of the resource. Here's how to define resources in your model's config file (`config.pbtxt`):
```txt
instance_group [
  {
    count: 1
    kind: KIND_GPU
    gpus: [ 0, 1, 2 ]
    rate_limiter {
      resources [
        {
          name: "R1"
          count: 4
        },
        {
          name: "R2"
          global: True
          count: 2
        }
      ]
      priority: 2
    }
  }
]
```
In this example:
- The model instance requires 4 units of resource "R1" and 2 units of resource "R2".
- "R2" is specified as a global resource, meaning it's shared across all devices.
- This configuration creates 3 model instances, one on each GPU (0, 1, and 2).
The three instances won't contend for "R1" among themselves as it's local to each device. However, they will contend for "R2" because it's a global resource shared across the system.
**Note:** When configuring the Rate Limiter, it's important to understand what the resource numbers means that from the pool of all resources (global or per-device), this model requires 4 R1 resources to execute.
For example, if you have 10 R1 resources for a device in total, and you have 3 instances of model A, each requiring 4 R1 resources, only two instances of model A can be executed simultaneously.
### Setting Total Resources
By default, the available number of resource copies is the maximum across all model instances that list that resource. For example, if you have three model instances with the following resource requirements:
```
A: [R1: 4, R2: 4]
B: [R2: 5, R3: 10, R4: 5]
C: [R1: 1, R3: 7, R4: 2]
```
Triton will create the following resources:
```
R1: 4
R2: 5
R3: 10
R4: 5
```
You can override these defaults using the `--rate-limit-resource` option when starting Triton:
```shell
docker run --gpus all --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v $(pwd)/models:/models nvcr.io/nvidia/tritonserver:21.09-py3 tritonserver --model-repository=/models --rate-limit-resource=R1:10 --rate-limit-resource=R2:5:0 --rate-limit-resource=R2:8:1 --rate-limit-resource=R3:2:global
```
This command sets up the following resources:
```
GLOBAL   => [R3: 2]
DEVICE 0 => [R1: 10, R2: 5]
DEVICE 1 => [R1: 10, R2: 8]
```
 #### `--rate-limit-resource` Flags:
 - **`--rate-limit-resource=R1:10`**:
     - **R1**: The name of the resource.
     - **10**: The limit for this specific resource.
 - **`--rate-limit-resource=R2:5:0`**:
     - **R2**: The resource name.
     - **5**: The limit for this specific resource.
     - **0**: The resource's instance index (e.g., the 0th instance of **R2**).
 - **`--rate-limit-resource=R2:8:1`**:
     - Similar to the above, this specifies that the 1st instance of **R2** has a limit of 8.
 - **`--rate-limit-resource=R3:2:global`**:
     - **R3**: Another resource.
     - **2**: Limit for this resource.
     - **global**: Instead of being instance-specific, this limit applies globally across all instances of this resource.
### Configuring Priorities:
Priority serves as a weighting value for scheduling across all model instances. 
An instance with priority 2 will be given half the number of scheduling chances as an instance with priority 1. You can set the priority in the rate limiter configuration:
```
rate_limiter {
  priority: 2
}
```
#### Example:
Imagine you are running two model instances on the same Triton server:
1. **Instance A** (latency-sensitive): Should handle requests faster, so you set `priority: 1`.
2. **Instance B** (less critical): Doesn't need as much scheduling attention, so you set `priority: 2`.
Here’s how you can configure this in Triton’s `config.pbtxt` file:
```protobuf
instance_group [
      {
        kind: KIND_GPU
    count: 1
    rate_limiter {
          priority: 1
    }
  },
  {
        kind: KIND_CPU
    count: 1
    rate_limiter {
          priority: 2
    }
  }
]
```
**Instance A (GPU)** with `priority: 1` will be scheduled twice as often as **Instance B (CPU)** with `priority: 2`, giving it more chances to process requests faster.
## 7. Practical Example: Configuring Rate Limiter for Multiple Models
Here is a practical example of configuring the Rate Limiter for multiple models.
### Step 1: Calculate Total GPU Memory
First, determine the total memory of your GPU. Let's say we have a GPU with 16GB (16,384 MB) of memory.
### Step 2: Calculate Base Memory Usage
Calculate the memory usage of each model when loaded into Triton (not serving any inference requests). For this example, let's assume:
- Model A: 500 MB
- Model B: 700 MB
- Model C: 600 MB
Total base memory usage: 500 + 700 + 600 = 1,800 MB
### Step 3: Calculate Available Memory
Subtract the base memory usage from the total GPU memory:
16,384 MB - 1,800 MB = 14,584 MB
This 14,584 MB is our available memory for inference (let's call this value m).
### Step 4: Determine Maximum Memory Usage
Find out the maximum memory usage of each model instance during inference. For this example, let's assume:
- Model A: 1,000 MB
- Model B: 1,500 MB
- Model C: 1,200 MB
### Step 5: Set Resource Values
Now, we'll set the R1 values for each model based on their maximum memory usage. We'll use 1 R1 unit to represent 100 MB of GPU memory:
```
instance_group [
  {
    count: 1
    kind: KIND_GPU
    rate_limiter {
      resources [
        {
          name: "R1"
          count: 10  # Model A: 1,000 MB / 100 MB per unit = 10 units
        }
      ]
    }
  }
]
```
Similarly, for Model B, count would be 15, and for Model C, it would be 12.
### Step 6: Set Total Per-Device Resource Value
Set the total per-device R1 value to m (our available memory for inference):
```shell
tritonserver --rate-limit-resource=R1:145 --model-repository=/path/to/model_repository
```
Here, 145 represents 14,584 MB / 100 MB per unit, rounded down.
### Complete Configuration Example
Here's how the complete configuration might look:
```
# Model A config.pbtxt
name: "model_a"
platform: "onnxruntime_onnx"
max_batch_size: 8
instance_group [
      {
        count: 1
    kind: KIND_GPU
    rate_limiter {
          resources [
            {
              name: "R1"
          count: 10
        }
      ]
    }
  }
]
# Model B config.pbtxt
name: "model_b"
platform: "onnxruntime_onnx"
max_batch_size: 8
instance_group [
      {
        count: 1
    kind: KIND_GPU
    rate_limiter {
          resources [
            {
              name: "R1"
          count: 15
        }
      ]
    }
  }
]
# Model C config.pbtxt
name: "model_c"
platform: "onnxruntime_onnx"
max_batch_size: 8
instance_group [
      {
        count: 1
    kind: KIND_GPU
    rate_limiter {
          resources [
            {
              name: "R1"
          count: 12
        }
      ]
    }
  }
]
```
With this configuration:
- Model A can use up to 1,000 MB of GPU memory during inference
- Model B can use up to 1,500 MB
- Model C can use up to 1,200 MB
 **Note:** The total usage across all models won't exceed 14,500 MB, leaving some buffer and accounting for the base memory usage

## 8. Best Practices and Tips
- Use the Model Analyzer (https://github.com/triton-inference-server/model_analyzer) to help with memory profiling steps.
- Monitor your system's performance and adjust resource allocations as needed.
- Remember to account for base memory usage of loaded models.
- Leave some buffer in your calculations to account for system overhead and variations in memory usage.
- For models with highly variable memory usage, consider setting the resource count to the maximum expected usage to ensure stability.
## 9. Advanced Rate Limiter Configurations
### Global vs. Per-Device Resources
- Per-device resources: By default, resources are per-device. Resource requirements for a model instance are enforced against the resources associated with the device where the model instance runs.
- Global resources: Instead of creating resource copies per-device, a global resource has a single copy shared across the entire system. You can specify a global resource in the model configuration or when starting Triton.
### Fine-tuning Priorities
In a resource-constrained system, there will be contention for resources among model instances. The priority setting helps determine which model instance to select for the next execution. Lower priority values indicate higher priority (more scheduling chances).

## 10. Conclusion
The Rate Limiter is a powerful tool in Triton that helps you:
- Manage resources efficiently
- Run multiple models safely
- Improve system stability

Triton is a tool that helps deploy and run machine learning models. It has a feature called the Rate Limiter, which helps manage resources better.

Rate Limiter in Triton: Managing Resource Allocation for MLOps

# Functools in Machine Learning
## Introduction:
You're working on a machine learning project:
Your model prediction function is slow because it performs the same calculations repeatedly for the same inputs.
- You've written a complex data preprocessing function, and you notice it's running the same operations on the same data multiple times, wasting time and resources.
- You've created a custom class to represent your data, but now you need to compare and sort instances of this class, which requires writing multiple comparison methods.
👉 This is where `functools` comes in.

## What is `functools`?
`functools` is a **module in Python's standard library** that provides **higher-order functions** and operations on **callable objects** (Functions, Classes, Built-in Functions, etc.) 
### Higher-Order Functions:
Higher-order functions are:
- functions that can either take other functions as arguments
- return functions as their results. 
![1.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/c360ce742dee6cf4d28249bfffad2881bad8e4d5c824a27a87466dd3ed636165.jpg)

Let's look at a simple example to understand this better:
```python
def apply_operation(func, x, y):
    return func(x, y)
def add(a, b):
    return a + b
def multiply(a, b):
    return a * b
# Using the higher-order function
result1 = apply_operation(add, 5, 3)
print(result1)  # Output: 8
result2 = apply_operation(multiply, 5, 3)
print(result2)  # Output: 15
```
In this example:
- `apply_operation` is a higher-order function because it takes another function (`func`) as an argument.
- We can pass different functions (`add` or `multiply`) to `apply_operation`, and it will use whichever function we provide.
#### Higher-Order Functions in Machine Learning example:
```python
import numpy as np
def apply_to_dataset(func, dataset):
    return func(dataset)
def normalize(dataset):
    return (dataset - np.mean(dataset)) / np.std(dataset)
def standardize(dataset):
    return (dataset - np.min(dataset)) / (np.max(dataset) - np.min(dataset))
# Sample dataset
data = np.array([1, 2, 3, 4, 5])
# Apply different preprocessing functions using apply_to_dataset
normalized_data = apply_to_dataset(normalize, data)
standardized_data = apply_to_dataset(standardize, data)
print("Normalized:", normalized_data)
print("Standardized:", standardized_data)
```
In this ML-related example, `apply_to_dataset` is a higher-order function that can apply any preprocessing function to our dataset.
We can easily switch between different preprocessing methods (`normalize` or `standardize`) without changing the main function.
### Key Functions in `functools`
#### 1. `functools.partial`
The `partial` function from the `functools` module is a powerful tool that allows you to "freeze" some portion of the function's arguments, creating a new function that you can call with fewer arguments. This is particularly useful for reusing functions with different fixed arguments.
Example:
```python
from functools import partial
def greet(greeting, name):
    return f"{greeting}, {name}!"
# Creating a new function 'hello' where the 'greeting' argument is pre-set to "Hello"
hello = partial(greet, "Hello")
# Now, you only need to pass the 'name' argument when calling 'hello'
print(hello("Alice"))  # Output: Hello, Alice!
print(hello("Bob"))    # Output: Hello, Bob!
```

- The `partial` function is used to create a new function, `hello`, which is a version of the `greet` function where the `greeting` argument is pre-set to "Hello".
- When you call `hello("Alice")`, it effectively calls `greet("Hello", "Alice")`.
- This is very useful when you need to reuse the same function but with some arguments already fixed.
##### Real-world use case:
For instance, if you are working with a mathematical function that frequently requires the same parameters, you can use `partial` to simplify your code and reduce redundancy.
#### 2. `functools.lru_cache`
`lru_cache` is a decorator that implements memoization, caching a function's results based on its arguments.
Example:
```python
from functools import lru_cache
# Counter to keep track of function calls
call_counter = 0
@lru_cache(maxsize=None)
def fibonacci(n):
    global call_counter
    call_counter += 1  # Increment the counter each time the function is called
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)
# Compute the Fibonacci number for 100
print(f"Fibonacci(100): {fibonacci(100)}")
# Verify how many times the function was called
print(f"Function was called {call_counter} times.")
```
Output:
```shell
Fibonacci(100): 354224848179261915075
Function was called 101 times.
```
In this case, `fibonacci(100)` is computed efficiently with only 101 calls to the function (instead of an exponentially large number without caching).
#### 3. `functools.cache`
`cache` is similar to `lru_cache` but provides simpler syntax for unlimited caching.
Example:
```python
from functools import cache
# Counter to track the number of function calls
call_counter = 0
@cache
def expensive_computation(x):
    global call_counter
    call_counter += 1  # Increment the counter each time the function is called
    # Simulate an expensive operation
    return x ** 2
# First call - function is computed
print(expensive_computation(10))  # Output: 100 (computed)
# Second call - result is retrieved from the cache
print(expensive_computation(10))  # Output: 100 (cached)
# Verify how many times the function was called
print(f"Function was called {call_counter} times.")
```
```shell
100  # Computed
100  # Retrieved from cache
Function was called 1 times.
```
**This output verifies that the second call retrieves the result from the cache rather than recomputing it.**
#### 4. `functools.total_ordering`
`total_ordering` is a class decorator that fills in missing comparison methods in a class that defines at least one comparison method.
Example:
```python
from functools import total_ordering
@total_ordering
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age
    def __eq__(self, other):
            return self.age == other.age
    def __lt__(self, other):
            return self.age < other.age
p1 = Person("Alice", 30)
p2 = Person("Bob", 25)
print(p1 > p2)  # True
```
## Why `functools` is Valuable in Machine Learning
In machine learning projects, we often deal with complex, computationally expensive operations, large datasets, and the need for flexible, reusable code. `functools` provides tools that can help address these challenges. 
Let's explore the main functions in `functools` and see how they can be applied in ML contexts.
### 1. `functools.partial`
**What**: `partial` creates a new function with some of the arguments of the original function pre-set.
**Why it's useful in ML**: 
- Simplifies the creation of specialized functions from more general ones
- Helps in creating consistent preprocessing or model configuration functions
**Example in ML**:
```python
from functools import partial
from sklearn.ensemble import RandomForestClassifier
# Create a partially configured RandomForest
rf_classifier = partial(RandomForestClassifier, 
                        n_estimators=100, 
                        random_state=42)
# Now you can create classifiers with different max_depths easily
clf_shallow = rf_classifier(max_depth=5)
clf_deep = rf_classifier(max_depth=15)
# This is especially useful in grid search or pipelines
from sklearn.model_selection import GridSearchCV
param_grid = {'max_depth': [5, 10, 15, 20]}
grid_search = GridSearchCV(rf_classifier(), param_grid, cv=5)
# You can fit this grid_search object to your data
# grid_search.fit(X, y)
```
In this example, `partial` allows us to create a base RandomForest configuration, making it easier to experiment with different `max_depth` values while keeping other parameters constant.
### 2. `functools.lru_cache`
**What**: `lru_cache` is a decorator that caches the results of a function, avoiding repeated computations for the same inputs.
**Why it's useful in ML**: 
- Speeds up expensive computations, especially in feature engineering or model inference
- Reduces redundant calculations in iterative processes like cross-validation
**Example in ML**:
```python
from functools import lru_cache
import numpy as np
@lru_cache(maxsize=100)
def expensive_feature_computation(data):
    # Simulate an expensive computation
    return np.mean(np.array(data) ** 2)
# In your feature engineering pipeline
def engineer_features(dataset):
    return [expensive_feature_computation(tuple(row)) for row in dataset]
# The function will only compute for unique rows, reusing cached results
dataset = [(1, 2, 3), (4, 5, 6), (1, 2, 3)]
features = engineer_features(dataset)
print(features)
```
Here, `lru_cache` ensures that the expensive feature computation is only done once for each unique input, significantly speeding up the feature engineering process for datasets with repeated or similar samples.
### 3. `functools.cache`
**What**: Similar to `lru_cache`, but with no size limit on the cache.
**Why it's useful in ML**: 
- Ideal for pure functions with a limited input domain
- Useful in recursive algorithms or dynamic programming approaches in ML
**Example in ML**:
```python
from functools import cache
@cache
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)
# This can be used in feature engineering or certain ML algorithms
def fib_feature(x):
    return fibonacci(int(x))
# Apply to a dataset
dataset = [5, 8, 13, 5, 8]
fib_features = [fib_feature(x) for x in dataset]
print(fib_features)
```
While this is a simple example, it demonstrates how `cache` can optimize recursive computations, which could be part of more complex feature engineering or custom ML algorithms.
### 4. `functools.total_ordering`
**What**: A class decorator that automatically generates ordering methods (`__lt__`, `__gt__`, etc.) based on `__eq__` and one other comparison method.
**Why it's useful in ML**: 
- Simplifies the creation of custom, comparable objects (e.g., for custom metrics or model results)
- Useful in sorting or ranking scenarios in ML pipelines
**Example in ML**:
```python
from functools import total_ordering
@total_ordering
class ModelPerformance:
    def __init__(self, accuracy, f1_score):
        self.accuracy = accuracy
        self.f1_score = f1_score
    
    def __eq__(self, other):
        return (self.accuracy, self.f1_score) == (other.accuracy, other.f1_score)
    
    def __lt__(self, other):
        return (self.accuracy, self.f1_score) < (other.accuracy, other.f1_score)
# Now you can easily compare model performances
model1 = ModelPerformance(0.85, 0.82)
model2 = ModelPerformance(0.82, 0.85)
print(model1 > model2)  # True
print(max(model1, model2))  # ModelPerformance(0.85, 0.82)
# This is useful for sorting model results
models = [ModelPerformance(0.8, 0.75), ModelPerformance(0.85, 0.8), ModelPerformance(0.82, 0.78)]
best_model = max(models)
print(f"Best model: Accuracy {best_model.accuracy}, F1 {best_model.f1_score}")
```
This example shows how `total_ordering` can be used to create a custom class for model performance that can be easily compared and sorted, which is often necessary in model selection or ensemble methods.

### 5. Putting It All Together: `functools` in a ML System:
Here's a simple example showing how you might use `functools` in different parts of a machine learning system:
```python
from functools import lru_cache, partial, cache
import numpy as np
from sklearn.ensemble import RandomForestClassifier
# Data handling
@cache
def load_data(file_name):
    # Pretend we're loading data from a file
    return np.random.rand(1000, 10)
# Model setup
base_model = partial(RandomForestClassifier, n_estimators=100, random_state=42)
# Model evaluation
@lru_cache(maxsize=10)
def evaluate_model(model, X, y):
    # Pretend we're doing cross-validation here
    return np.mean(model.predict(X) == y)
# Prediction function
@lru_cache(maxsize=1000)
def predict(model, features):
    return model.predict([features])[0]
# Using these functions
X = load_data("fake_data.csv")
y = np.random.randint(0, 2, 1000)  # Fake labels
model = base_model(max_depth=5)
model.fit(X, y)
print(f"Model score: {evaluate_model(model, tuple(map(tuple, X)), tuple(y))}")
# Pretend API call
print(f"Prediction: {predict(model, tuple(X[0]))}")
```
## Conclusion
Throughout this tutorial, we've explored how the `functools` module can be a powerful ally in your machine learning and MLOps workflows. Let's recap the key takeaways:
1. **Efficiency Boost**: Functions like `lru_cache` and `cache` can significantly speed up your ML pipelines by avoiding redundant computations. This is particularly valuable in scenarios involving expensive feature engineering or repeated model inferences.
2. **Flexibility in Code**: `partial` allows you to create specialized versions of functions, making it easier to experiment with different model configurations or preprocessing steps. This flexibility is crucial in the iterative nature of ML development.
3. **Simplified Comparisons**: `total_ordering` simplifies the creation of custom, comparable objects, which can be invaluable when dealing with complex model metrics or custom performance measures.
4. **Cleaner, More Maintainable Code**: By leveraging these tools, you can write more modular and reusable code, which is essential in managing the complexity of ML projects.
5. **Optimized Resources**: Proper use of `functools` can lead to more efficient use of computational resources, a critical factor when working with large datasets or complex models

## References:
[1] https://docs.python.org/3/library/functools.html

[2] https://www.geeksforgeeks.org/functools-module-in-python/

[3] https://towardsdatascience.com/introducing-pythons-functools-module-2c4cba4774e


`functools` is a **module in Python's standard library** that provides **higher-order functions** and operations on **callable objects** (Functions, Classes, Built-in Functions, etc.) 

 Functools in Machine Learning

# Airflow Macros and Jinja: Enhancing Machine Learning Workflows
## 1. Introduction:
Apache Airflow is a powerful tool for creating and managing data pipelines. In the world of Machine Learning (ML), where data processing and model training often involve complex, multi-step workflows, **Airflow shines as a way to organize and automate these processes**. For example:
![1.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/77cd873b1b2506602c861df5b09a7456ba03d5f1ab829aabf86ca423de07bbaa.jpg)
One of the biggest challenges in ML workflows is ***the need for flexibility***:
- Data can change
- Models need updating
- parameters require tuning
- etc.
**This is where Airflow's dynamic pipelines come in handy.** 
👉 They allow us to create workflows that can adapt to changing conditions without needing to rewrite our code constantly.
## 2. Airflow Macros: The Powerhouse of Dynamic Workflows
Airflow macros are pre-defined variables or functions that you can use in your Airflow tasks. They act like **placeholders** that get filled with real values when your task runs. 
This means you can write code once that works for different dates, times, or other changing values.
![2.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/1d4deef3578bc7e247b7991be859b0985fc677d992ae2a293c05becaab9c83ae.jpg)

**Note:**
- Imagine you have a pipeline that needs to process yesterday's data every day. Instead of changing the date manually each time, you can use a macro that automatically gives you yesterday's date. This makes your pipeline dynamic and saves you from potential errors.
### Jinja templating: The foundation for macros
Airflow uses a system called Jinja templating to make macros work. Jinja is like a smart find-and-replace tool. It looks for special markers in your code (usually wrapped in double curly braces like this: `{{ }}`) and replaces them with actual values when the code runs.
![3.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/1d742d50cb87e1df9579885c33c86d7b87c08c273930da55c2eb7025dfa0154f.jpg)
## 3. Essential Airflow Macros for ML Pipelines
Let's look at some of the most useful macros for ML workflows:
### Datetime macros for time-based operations
- `{{ ds }}`: This gives you the date of the current run in the format 'YYYY-MM-DD'. It's super useful for processing data for a specific day.
- `{{ yesterday_ds }}`: As the name suggests, this gives you yesterday's date. Perfect for when you need to look at recent data.
### ds, ds_nodash, ts, and their applications
- `{{ ds }}`: We mentioned this earlier. It's great for naming files or folders by date.
  Example: `output_{{ ds }}.csv` might create a file named `output_2023-09-14.csv`.
- `{{ ds_nodash }}`: This is like `ds`, but without the dashes. Useful when you can't have dashes in names.
  Example: `data_{{ ds_nodash }}.parquet` could create `data_20230914.parquet`.
- `{{ ts }}`: This gives you the current timestamp. It's handy for very precise timing or creating unique identifiers.
  Example: `model_{{ ts }}.pkl` might save a model as `model_2023-09-14T10:30:00+00:00.pkl`.
### params and their use in parametrizing DAGs
The `params` macro lets you pass custom parameters to your tasks. This is incredibly useful for making your DAGs flexible.
Example:
```python
default_args = {
        'owner': 'airflow',
    'start_date': datetime(2023, 1, 1),
    'params': {
            'model_type': 'random_forest',
        'n_estimators': 100
    }
}
with DAG('ml_training_dag', default_args=default_args) as dag:
    train_model = PythonOperator(
            task_id='train_model',
        python_callable=train_model_func,
        op_kwargs={
                'model_type': '{{ params.model_type }}',
            'n_estimators': '{{ params.n_estimators }}'
        }
    )
```
In this example, we can easily change the model type or number of estimators without changing the DAG code itself.
### Other useful macros
- `{{ macros.ds_add(ds, days) }}`: This lets you add or subtract days from a given date. Useful for creating date ranges or looking at specific time periods.
- `{{ task_instance }}`: This gives you access to the current task instance, which can be helpful for getting runtime information or previous task results.
- `{{ dag }}`: This provides access to the DAG object, allowing you to get information about the DAG itself.
### Example ML Workflow with Macros:
![4.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/5cfa8dfce6403ae4a2fbf0c722b5d18491a53446330d6a9f9f0fb27e5304bda9.jpg)

**In this image:**
- The **Fetch Data** task dynamically pulls the data for the day using the `{{ ds }}` macro.
- The **Process Data** task uses `{{ yesterday_ds }}` to process data from the previous day.
- The **Train Model** task uses the `{{ params.model_type }}` and `{{ params.task_instance }}` macros to train the specified model type with instance-specific parameters.
- The **Deploy Model** task ensures that the model is deployed based on the model's instance data.
👉 This shows how macros and Jinja templating help in creating flexible and dynamic pipelines that adapt to different conditions without needing constant manual updates.
## 4. Creating Custom Macros
### How to create Custom Macros
You can create your own macros to encapsulate complex logic or frequently used operations. Here's how:
1. Define your macro function in a Python file (e.g., `custom_macros.py`):
```python
def days_ago(n):
    from datetime import datetime, timedelta
    return (datetime.now() - timedelta(days=n)).strftime('%Y-%m-%d')
```
2. Add the path to this file in your Airflow configuration (`airflow.cfg`):
```
user_defined_macros = /path/to/custom_macros.py
```
3. Use your custom macro in your DAG:
```python
task = BashOperator(
    task_id='print_date',
    bash_command='echo "Date from 7 days ago: {{ days_ago(7) }}"',
    dag=dag
)
```
### Custom Macros for ML-specific Operations
Let's create a macro for data normalization:
```python
def normalize_column(column_name, min_val, max_val):
    return f"""
    ({column_name} - {min_val}) / ({max_val} - {min_val}) AS normalized_{column_name}
    """
# In your DAG:
normalize_task = BigQueryOperator(
        task_id='normalize_data',
    sql=f"""
    SELECT 
        {{ normalize_column('feature1', 0, 100) }},
        {{ normalize_column('feature2', -50, 50) }}
    FROM `my_dataset.my_table`
    """,
    use_legacy_sql=False,
    dag=dag
)
```
This macro helps standardize the process of normalizing columns in your dataset.
## 5. Airflow Macros in Machine Learning Workflows
### Use Case: Automated Hyperparameter Tuning
We can use macros to dynamically set hyperparameters:
```python
def set_learning_rate(base_rate, decay_factor):
    return f"""
    {base_rate} * pow({decay_factor}, {{{{ task_instance.try_number - 1 }}}})
    """
train_model_task = PythonOperator(
        task_id='train_model',
    python_callable=train_model_func,
    op_kwargs={
            'learning_rate': "{{ set_learning_rate(0.1, 0.9) }}"
    },
    dag=dag
)
```
This macro adjusts the learning rate based on the number of task attempts, implementing a simple learning rate decay.
## 6. Advanced Techniques: Combining Macros with Airflow Operators
### PythonOperator with macros
```python
def process_data(date, feature_count):
    # Your data processing logic here
    pass
process_task = PythonOperator(
        task_id='process_data',
    python_callable=process_data,
    op_kwargs={
            'date': '{{ ds }}',
        'feature_count': '{{ task_instance.xcom_pull(task_ids="count_features") }}'
    },
    dag=dag
)
```
This example shows how to use macros to pass dynamic values to a Python function.
### BranchPythonOperator with macros
```python
def choose_model(ti):
    data_size = ti.xcom_pull(task_ids='check_data_size')
    if data_size > 1000000:
        return 'train_large_model'
    else:
        return 'train_small_model'
branch_task = BranchPythonOperator(
        task_id='choose_model',
    python_callable=choose_model,
    dag=dag
)
train_large_model = PythonOperator(
        task_id='train_large_model',
    python_callable=train_large_model_func,
    op_kwargs={'date': '{{ ds }}'},
    dag=dag
)
train_small_model = PythonOperator(
        task_id='train_small_model',
    python_callable=train_small_model_func,
    op_kwargs={'date': '{{ ds }}'},
    dag=dag
)
branch_task >> [train_large_model, train_small_model]
```
This example uses a BranchPythonOperator to dynamically choose which model to train based on the data size.
## 7. Best Practices and Conclusion
### Tips for effective use of macros in ML workflows
1. Keep macros simple and focused on a single task.
2. Use descriptive names for custom macros to enhance readability.
3. Document your macros, especially if they contain complex logic.
4. Test your macros thoroughly to ensure they behave as expected in different scenarios.
### Potential pitfalls and how to avoid them
1. Overuse of macros can make DAGs hard to read. Use them judiciously.
2. Be cautious with macros that access external resources, as they may slow down DAG parsing.
3. Remember that macros are evaluated at runtime, not during DAG parsing. This can lead to unexpected behavior if not considered.
## References:
[1] https://airflow.apache.org/docs/apache-airflow/1.10.12/macros-ref.html

[2] https://medium.com/@laxmirathaur11/airflow-beyond-basics-part-4-6215e8e5d6be

Apache Airflow is a powerful tool for creating and managing data pipelines. In the world of Machine Learning (ML), where data processing and model training often involve complex, multi-step workflows, **Airflow shines as a way to organize and automate these processes**. For example:

Airflow Macros and Jinja: Enhancing Machine Learning Workflows

The big question is: What can we do with this dataset? How can we use it to evaluate company performance and deliver insights for improvement?

Identify and Measure Key Metrics to Analyze the Company’s Performance

# Edit Your Images Easily with Inpainting and Diffusion

Wondering how new technology supports us in image modification and editing? It turns out **diffusion** can help us a lot. Among these techniques, **image-to-image** methods like **impainting** are especially useful. Instead of generating a whole new image, we focus only on modifying specific parts of an existing image.
![example_mug.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/f16d2fe39613591d49925714f12473289fa989753b0677faab39f767a88e36f5.jpg)
Whether using **Stable Diffusion** or other variants, each can demonstrate benefits in specific fields.
In this tutorial, we will guide you through:
- Preparing the environment  
- Installing the library  
- Loading the model and performing inference in Python  
- Creating an interactive UI with **Gradio** for a better experience  
![example_necklake.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/46a2de1e80e17f8643ca6fb2ae7c63d9b048fbe53ff40c64a8c15401027d68a3.jpg)
---
**Requirements**: To run this lab, we suggest having a **GPU** with 12GB of VRAM, or using **Google Colab** is preferred.

---
## Prepare the Environment
In this tutorial, we run the **diffusion model** in a **Python 3.9** environment using the **Anaconda** manager. If you are using Google Colab, you can skip this step.
Below is how you can set up the environment:
```bash
# Create a new conda environment
conda create -n diffusion python=3.9
conda activate diffusion
```
Then, we need to install **Torch** and **Transformers**. These libraries are essential for running the diffusion model.
```bash
# Install the transformers library
pip install transformers
```
Pay attention to the version of **CUDA** on your system. If you are using Google Colab, you don’t need to install **Torch**. For example, in our system, we are using **CUDA 12.4** and above, so the installation will be:
```bash
# Install torch based on your CUDA version
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu124
```
Next, we will install **Diffusers**, the module that enables inpainting.
```bash
# Install the diffusers library
pip install diffusers
```
---
## Load the Model
If you check out **Hugging Face**, several models are available. Remember to select the **impainting model**.
In this tutorial, let's use **diffusers/stable-diffusion-xl-1.0-inpainting-0.1** as our starting point.
The code to load a pipeline that applies this model is as follows:
```python
import torch
from diffusers import AutoPipelineForInpainting
# Load model and move it to GPU
pipe = AutoPipelineForInpainting.from_pretrained(
        "diffusers/stable-diffusion-xl-1.0-inpainting-0.1", 
    torch_dtype=torch.float16
).to("cuda")
```
**Explanation of the code above**:  
- We import **torch** and the **AutoPipelineForInpainting** from the **Diffusers** library.  
- The `from_pretrained` method loads the specified model, and we set it to run on the **GPU** using `torch.float16` for better performance.
---
## Load Image and Mask
The idea of **impainting** is to modify parts of an image.  
**Images** are the visuals we want to edit, while **masks** specify the areas we want to change.
We will load an available image and mask for testing as shown below:
```python
# Load an image to inpaint
url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint.png"
response = requests.get(url)
image = Image.open(BytesIO(response.content)).convert("RGB")
# Load the mask image
mask_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint_mask.png"
mask_response = requests.get(mask_url)
mask_image = Image.open(BytesIO(mask_response.content)).convert("RGB")
# Resize images if necessary
image = image.resize((1024, 1024))
mask_image = mask_image.resize((1024, 1024))
```
**Explanation of the code**:  
- We load the image and the mask using URLs.  
- The images are converted to **RGB** format and resized to **1024x1024 pixels**.
Let’s visualize the images we loaded from the above URLs:
```python
# Visualize both images in the same chart using matplotlib 
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].imshow(image)
ax[0].set_title("Image")
ax[0].axis("off")
ax[1].imshow(mask_image)
ax[1].set_title("Mask")
ax[1].axis("off")
plt.show()
```
![1.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/687275847dfbdadd4b50c7282b696d78907422d337a1affdfdd4119b778d4572.jpg)

As you can see, the original image shows a picture of a road and a mountain in the center. The mask covers the part of the mountain. Our aim is to use **impainting** to change this mountain using a prompt.

---
## Set Up the Prompt
A **prompt** is a textual description of what we want to generate in the masked area.
```python
generator = torch.Generator("cuda").manual_seed(92)
prompt = 'an elven castle, in the mountain mist'
```
We use a **generator** to ensure the output is consistent whenever we use the same prompt.
Next, we input the prompt, image, and mask into the pipeline:
```python
# Generate the inpainted image
impainted_image = pipe(prompt=prompt, image=image, mask_image=mask_image, generator=generator).images[0]
```
It may take some time to process, depending on the speed of your GPU.
Let’s visualize the results of inpainting:
```python
# Use matplotlib to visualize the image, mask, and inpainted image
fig, ax = plt.subplots(1, 3, figsize=(15, 5))
ax[0].imshow(image)
ax[0].set_title("Image")
ax[0].axis("off")
ax[1].imshow(mask_image)
ax[1].set_title("Mask")
ax[1].axis("off")
ax[2].imshow(impainted_image)
ax[2].set_title("Inpainted Image")
ax[2].axis("off")
plt.show()
```
![2.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/68a7eb4fa32fb99c3510a5a1227fcbc90bf4eeb1929bcf5a7213a0963f4d972c.jpg)
You can see that the area we masked before has been changed, while the rest of the image remains intact.

---
## Change the Parameters
We can adjust the parameters in the pipeline, such as **guidance scale**, **number of inference steps**, and **strength**.
```python
impainted_image = pipe(
        prompt=prompt,
    image=image,
    mask_image=mask_image,
    guidance_scale=8.0,
    num_inference_steps=20,  # Steps between 15 and 30 work well for us
    strength=0.99,  # Ensure `strength` is below 1.0
).images[0]
```
**Explanation of each parameter**:  
- **guidance_scale**: This controls how closely the generated image should match the prompt. Higher values make the image adhere more closely to the prompt.  
- **num_inference_steps**: This determines how many steps the model takes to generate the image. More steps can lead to better results.  
- **strength**: This adjusts how much of the original image is preserved. A value below 1.0 means some original features are retained.
  
---
## Interactive UI with Gradio
We can use **Gradio** to create an AI tool that allows users to draw a mask and apply inpainting.
```bash
# Install Gradio
pip install gradio
```
First, we create a function to receive the image, prompt, mask, and parameters:
```python
# Dummy inpainting function with hyperparameters
def impaint(img, prompt, num_steps, guidance_scale, strength):
    # Extract alpha channel to create a mask for inpainting
    background = img["background"][:, :, :3]  # Discard the alpha channel, keep RGB

    # Extract alpha channel to create a mask for inpainting
    alpha_channel = img["layers"][0][:, :, 3]
    mask = np.where(alpha_channel == 0, 0, 255).astype(np.uint8)

    background = Image.fromarray(background)    
    mask = Image.fromarray(mask)

    # Apply inpainting with the given prompt and hyperparameters
    impainted_image = pipe(prompt=prompt, image=background, 
                            mask_image=mask,
                            num_steps=num_steps, guidance_scale=guidance_scale, strength=strength
                            ).images[0]

    return impainted_image
```
- The **impaint** function takes in an image, prompt, and parameters.  
- It extracts the background and the alpha channel to create a mask.  
- The inpainting is performed with the given parameters, and the inpainted image is returned.
Next, we set up the Gradio interface:
```python
with gr.Blocks() as demo:

    with gr.Row():
        # Textbox to enter the inpainting prompt
        prompt_input = gr.Textbox(label="Inpainting Prompt", placeholder="Enter text prompt for inpainting")

    with gr.Row():
        # Numeric inputs for editing hyperparameters
        num_steps = gr.Slider(minimum=1, maximum=100, value=50, step=1, label="Number of Inference Steps")
        guidance_scale = gr.Slider(minimum=1.0, maximum=20.0, value=7.5, step=0.1, label="Guidance Scale")
        strength = gr.Slider(minimum=0.0, maximum=1.0, value=0.5, step=0.1, label="Strength")

    with gr.Row():
        img = gr.ImageEditor(
                crop_size="1:1",
            height="30vw"
        )
        im_preview = gr.Image(height="30vw")

    # Automatically process the uploaded image and show it immediately    
    btn = gr.Button("Process Image")
    btn.click(impaint, [img, prompt_input, num_steps, guidance_scale, strength], [im_preview])
    ```
- We set up the **Gradio** interface with input fields for the prompt and hyperparameters.  
- The **ImageEditor** allows users to draw the mask directly on the image.  
- When the **Process Image** button is clicked, the **impaint** function processes the inputs and displays the inpainted image.
Finally, we run the code:
```python
demo.launch()
```
To access the web app, go to: http://127.0.0.1:7860 in the browser. You can see the app interface as shown below:
![web.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/945119d1940ec1207c5c25fa26d27961718735a6d7a40a6653fcfc49a1c8a79c.jpg)
In this interface, you can change a variety of input options and upload your own image.
After uploading, you can use the drawing tool to draw a mask.  
![draw.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/11bd8fc78cb90d273af9a74813b4c5a00fdff7477efc3b27b5f12ba55e97b466.jpg)
After that, type in the form, adjust the values of parameters, and click the **Process Image** button.
![running.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/44bf66d7f841206a1cb692b5490f7b16135a6b15ae1664a03f1c6a5ae1bf7962.jpg)
After some time (depending on your GPU), we can view the result in the right panel.
![result.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/e2c0d0bb730b8c746e6b7e135c30e63f83bfcc455f6419d61d50f01439fd8d1c.jpg)
You can also change the prompt and other parameters to generate your desired output:
![result 2.jpg](https://fsds-content-prod.s3.ap-southeast-1.amazonaws.com/uploads/d9a1ac55c0e8ca1650095fd525262bafad89126044d80d7714beecd625e79daa.jpg)
Finally, you can save the output image by clicking the download button.

---
## Conclusion
In this tutorial, we explored how to use **impainting** with **diffusion models** to modify specific parts of an image effectively. With libraries like **Diffusers** and **Gradio**, we can create interactive applications that make image editing easy and accessible.
Feel free to experiment with different prompts and parameters to see how they affect the inpainting results!

---
## References
Hugging Face. (n.d.). Inpainting. In *diffusers documentation*. Retrieved December 16, 2024, from [https://huggingface.co/docs/diffusers/using-diffusers/inpaint](https://huggingface.co/docs/diffusers/using-diffusers/inpaint)

Wondering how new technology supports us in image modification and editing? It turns out **diffusion** can help us a lot. Among these techniques, **image-to-image** methods like **impainting** are especially useful. Instead of generating a whole new image, we focus only on modifying specific parts of an existing image.

FSDS Blog