Arm Create Hands-onOptimize WideResNet-50 for Alif E8

This page is for the Arm AI Model Optimizer, built on NetsPresso hands-on session at Arm Create 2026 only. Everything you need for the session is already set up on the server: the model, the datasets and the server address. Follow the steps in order and copy the commands from here.

What you'll build: WideResNet-50 optimized for the Alif Ensemble E8 (Cortex-M55 + Ethos-U85 NPU), with latency, memory and Top-1 accuracy measured before and after optimization, and an ExecuTorch .pte file you can take home.

Time: about 50 minutes, 10 steps.

Note: How this page differs from the general User Guide

The general User Guide is written for any deployment. For this session:

• No environment variables. The server address is built into the CLI. Skip the .env / export NP_... setup you may see elsewhere.
• No dataset upload. Calibration and evaluation datasets are preset for this session. Skip np datasets register.
• Token from the sign-up desk. You don't need an Arm AI Portal account today.

Before You Start

Item	What you need
Laptop	macOS or Linux. On Windows, see Windows users
Python	3.11 or later (python3 --version)
Install command	python -m pip install arm-ai-model-optimizer (from PyPI, the same on every platform)
Access token	Printed on the card from the sign-up desk
Network	Venue Wi-Fi. Installing pulls packages from public PyPI

Today's settings

You will type these values in Step 4. They are the same for everyone.

Item	Value
Model	wideresnet50_cls_14_pt2_pytorch_static_fp32 (WideResNet-50, image classification, preset)
Device	alif_e8_devkit (Alif Ensemble E8 DevKit)
Runtime	executorch
Backend	ethos_u
Calibration dataset	openimages_filtered_calib_seed42_n98 (Open Images V7, 98 images)
Evaluation dataset	openimages_filtered_eval_seed42_n162 (Open Images V7, 162 images)
Metric	Top-1 accuracy

Windows users

The commands on this page are written for bash. Which terminal you use decides how much changes:

• WSL2: follow this page as is.

• Git Bash: follow this page with the changes below.

  Where	This page	Git Bash
  Step 1, create the venv	python3 -m venv .venv	py -3.11 -m venv .venv (or python -m venv .venv)
  Activate the venv	source .venv/bin/activate	source .venv/Scripts/activate
  Paths the CLI prints	/.../arm-create/workspace	The CLI prints Windows paths such as C:\Users\you\arm-create\workspace. Put them in single quotes (cd 'C:\Users\you\...') or use forward slashes (cd /c/Users/you/...)

• PowerShell: every np command is the same, but the shell syntax around it differs:

  • Allow the venv script once with Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned, then activate with .\.venv\Scripts\Activate.ps1.
  • Install the CLI with the same command as on macOS and Linux: python -m pip install arm-ai-model-optimizer.
  • Multi-line commands continue with a backtick (`) instead of \, or type them on one line.
  • cat > file <<'EOF' blocks do not work. Open the file with notepad run.yaml, paste the YAML between <<'EOF' and EOF, and save.
  • Windows PowerShell 5.1 does not accept &&. Run those commands one per line.
  • Edit run.detail.yaml in Step 9 with Notepad instead of sed.

  If you can, use Git Bash or WSL2 instead; there are fewer changes to make.

Step 1. Install the CLI

Create a virtual environment and install the CLI from PyPI. The install command is the same on every platform:

mkdir arm-create && cd arm-create
python3 -m venv .venv
source .venv/bin/activate
python -m pip install arm-ai-model-optimizer

Check the install:

np --version

You should see

arm-ai-model-optimizer 2.1.6

Note: Opened a new terminal tab later? Run source .venv/bin/activate again from the arm-create folder, or np will not be found.

Step 2. Log In

Check that the server is reachable, then log in with the token from your card:

np health ready
np auth login <your-token>
np auth status

Replace <your-token> with the full token string from the card. It is long: copy it in one piece, with no line breaks.

You should see Access token valid in the summary:

[ Result Summary ]
 Auth server      https://auth.netspresso.staging.devplatform.arm.com
 User             <your-user>
 Tenant           <your-tenant>
 Access token     valid
 Access expires   <date and time>
 Refresh expires  <date and time>

You log in once. Access expires is about 30 minutes after login; the CLI renews the access token for you until Refresh expires, so you don't need to log in again during the session.

Step 3. Create a Workspace

np workspace init
cd workspace

You should see

[INFO ] [CLI] Workspace initialized successfully.

[ Result Summary ]
 Workspace Path     /.../arm-create/workspace
 Sample Experiment  /.../arm-create/workspace/sample_project/experiment_00
 ...

init also creates a sample_project. We won't use it today; you'll build your own project in the next step.

Step 4. Create a Project for Alif E8

A project ties one model to one target device and its datasets. Copy the whole command:

np projects create \
    --name wrn50-alif \
    --model wideresnet50_cls_14_pt2_pytorch_static_fp32 \
    --device alif_e8_devkit \
    --runtime executorch \
    --backend ethos_u \
    --calibration-dataset openimages_filtered_calib_seed42_n98 \
    --evaluation-dataset openimages_filtered_eval_seed42_n162

You should see

[DONE ] [CLI] Project created successfully.

[ Result Summary ]
  Project      wrn50-alif
  Model        wideresnet50_cls_14_pt2_pytorch_static_fp32
  Device       alif_e8_devkit
  Runtime      executorch
  Backend      ethos_u
  Calibration  openimages_filtered_calib_seed42_n98
  Evaluation   openimages_filtered_eval_seed42_n162

Prefer typing values one by one?

Run np projects create with no options. The CLI asks for each value; answer with the values in Today's settings.

Curious what else is available? These commands list it:

np models ls --filter domain=CV
np devices ls --filter backend=ethos_u
np datasets ls --filter type=PRESET

Step 5. Set Up the Experiment and Check It

An experiment holds one pipeline configuration (run.yaml). Create one with two steps, Graph Optimize (go) and Graph Quantize (gq):

cd wrn50-alif
np experiments create --steps go,gq
cd experiment_00

The generated run.yaml measures SNR only, so the report would have no accuracy column. Replace it with this version, which also measures Top-1 accuracy on the evaluation dataset for the original model, the Graph Optimize output and the final model. Copy the whole block:

cat > run.yaml <<'EOF'
version: '1.0'
steps:
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8
    automatic_mixed_precision: null
profile:
- INPUT_MODEL
- GRAPH_OPTIMIZE
- OUTPUT_MODEL
evaluate:
- model: INPUT_MODEL
  metric:
  - TOP1_ACCURACY
- model: GRAPH_OPTIMIZE
  metric:
  - SNR
  - TOP1_ACCURACY
- model: OUTPUT_MODEL
  metric:
  - SNR
  - TOP1_ACCURACY
EOF

Check the configuration before running it:

np validate

You should see

[DONE ] [CLI] Validation Passed. No issues found.
╭──────────────── Execution Plan ────────────────╮
│ Steps:                                          │
│   [1/2] graph_optimize                          │
│   [2/2] graph_quantize (scheme: W8-A8)          │
│ ...                                             │
╰─────────────────────────────────────────────────╯

What the two steps do

Step	What happens
Graph Optimize	Rewrites and fuses operators so Ethos-U85 runs them efficiently (for example, BatchNorm folded into Conv)
Graph Quantize	Converts FP32 weights and activations to INT8, using the 98 calibration images

After these two steps the server also evaluates accuracy and estimates latency and memory automatically.

How the optimization steps work (optional reading)

There are three optimization steps. This session runs the last two on a CV model; the Bonus runs all three on an LLM.

 Original    ┌───────────────────┐   ┌──────────────────┐   ┌──────────────────┐
  model  ───▶│ Advanced Quantize │──▶│  Graph Optimize  │──▶│  Graph Quantize  │───▶ model.pte
             │ adjust weights    │   │ rewrite graph    │   │ float → INT8     │
             └───────────────────┘   └──────────────────┘   └──────────────────┘
 CV  (WRN-50)      skipped                 fuse Conv+BN+ReLU       static scales
  starts FP32      -                       FP32                    FP32 → INT8
 LLM (Qwen)        AWQ                     reshape attention       dynamic scales
  starts FP16      FP16                    FP16                    FP16 → INT8

WideResNet-50 starts from an FP32 model and the LLM from an FP16 model. Both stay in that precision until Graph Quantize.

Step	In one line	Effect on accuracy
Advanced Quantize	Adjusts weight values so they lose less when rounded	Reduces the loss
Graph Optimize	Changes the graph structure, not what it computes	Practically none
Graph Quantize	Converts the numbers to INT8	This is where the loss comes from

1. Advanced Quantize: prepare the weights for rounding

CV (WideResNet-50): not used. Graph Optimize and Graph Quantize alone keep the accuracy loss small (73.98 → 73.47 with the tuned recipe in Step 9).

LLM (Bonus): AWQ. LLMs lose more accuracy from rounding, especially with 4-bit weights, which have only 16 levels. AWQ (Activation-aware Weight Quantization) reduces that loss.

A weight is multiplied by its input. If the input is large, a small rounding error in the weight becomes a large error in the output:

 Plain rounding (step 0.1)

   input        weight            rounded
   x1 =  1  ──▶ × 0.70  ──▶ 0.7  ─┐
                                  ├──▶ 0.7×1 + 0.1×20 = 2.70
   x2 = 20  ──▶ × 0.13  ──▶ 0.1  ─┘        exact 3.30 → error 0.60
                  ▲
                  loses only 0.03, but ×20 turns it into 0.60

AWQ finds these channels on the calibration data, multiplies their weights by a factor s before rounding, and divides their inputs by the same s. Because w × x = (w × s) × (x / s), the output does not change, but the larger weight is rounded with a smaller relative error:

 AWQ (s = 4 on channel 2)

   input          weight × s          rounded
   x1 =  1    ──▶ × 0.70         ──▶ 0.7  ─┐
                                           ├──▶ 0.7×1 + 0.5×5 = 3.20
   x2 / 4 = 5 ──▶ × 0.13×4=0.52  ──▶ 0.5  ─┘        exact 3.30 → error 0.10
     ▲
     ÷ s is merged into the layer before it, so no operator is added

A larger s is not always better: if the scaled weight becomes the largest in the tensor, the step widens for every other weight. AWQ tries several values of s for each layer and keeps the one with the smallest output error.

Setting (Bonus recipe)	Meaning
calibration.num_samples: 30	Calibration documents used to measure input sizes and test each s
clip: true	Also tries trimming the largest weights slightly, which makes the step finer for the others

In the report, 01_advanced_quantize is still FP16, so its PTE file is large (2522 MB). It shrinks after Graph Quantize.

2. Graph Optimize: same computation, a graph the hardware runs better

Each rule (a pattern handler) finds one pattern and replaces it with an equivalent one. A rule that finds nothing is skipped, so the rule list in run.detail.yaml is the same for many models.

CV (WideResNet-50): fuse the Conv blocks. A CNN repeats Conv → BatchNorm → ReLU, so fusing them matters most:

 Before                                        After
 x ─▶ Conv ─▶ BatchNorm ─▶ ReLU ─▶            x ─▶ Conv+ReLU ─▶
      (W,b)   (γ,β,μ,σ)                             (W',b')
      3 operators                                   1 operator, same output

 FuseBNToConv:  W' = W·γ/σ,   b' = (b−μ)·γ/σ + β

Rule	What it does
FuseBNToConv, CombineConvBNRelu	Merge Conv, BatchNorm and ReLU into one operator
RemoveDropout	Delete Dropout, which does nothing at inference
CastInt64InputsToInt32	Convert int64 to int32, which embedded backends support

LLM (Bonus): reshape the attention. A transformer is mostly attention and Linear layers:

 DecomposeScaledDotProductAttention

 Before                      After
 Q,K,V ─▶ [ SDPA ] ─▶ out    Q ─┐
                             K ─┴▶ MatMul ─▶ ×scale ─▶ +mask ─▶ Softmax ─┐
                             V ──────────────────────────────────────────┴▶ MatMul ─▶ out
                             each step can run on the backend and be quantized

 FuseQKV

 Before                          After
 x ─┬─▶ Linear_Q ─▶ Q
    ├─▶ Linear_K ─▶ K            x ─▶ Linear_QKV ─▶ split ─▶ Q, K, V
    └─▶ Linear_V ─▶ V
    3 small matmuls              1 large matmul

Rule	What it does
DecomposeScaledDotProductAttention	Split one attention operator into its steps
FuseQKV	Merge the Q, K and V Linear layers into one. The Alif E8 recipe has the opposite rule, SplitQKV, because each target runs a different form best
ReplaceMatmulWithLinear	Turn a MatMul with constant weights into a Linear, so its weights get quantized

In the report, 01_graph_optimize keeps the original Top-1 and has an SNR of about 200: its output is practically identical.

3. Graph Quantize: floating point to INT8

The model arrives in the precision it started with: FP32 for WideResNet-50, FP16 for the LLM. INT8 has 256 values, so each tensor gets a scale that maps its real range onto 256 steps:

 float  0.0                  1.50                                    6.0
         |--------------------x---------------------------------------|
 INT8    0   1   2   ...     64                                      255

         scale = 6.0 / 255 ≈ 0.0235
         1.50 → stored as 64 → read back 64 × 0.0235 ≈ 1.506   (error ≈ 0.006)

Weights are fixed, so their range is read from the model. Activations depend on the input, and CV and LLM differ in when their range is set.

CV (WideResNet-50): static scales (W8-A8). The ranges are measured once on the calibration images and fixed. Ethos-U supports only this method.

 On the server, once                                    On the board, every inference
 98 calibration images ─▶ record ranges ─▶ fix scales ═▶ use the fixed scales

Setting (Step 9 recipe)	Meaning
weight TENSOR + MINMAX	One scale per weight tensor, from its actual min and max
activation TENSOR + HISTOGRAM	One scale per activation, picked from a histogram of the calibration results

LLM (Bonus): dynamic scales (W8-A8_dynamic). Within one layer, the size of the values changes a lot from token to token, so a fixed scale wastes steps on small tokens. The activation scale is computed for each token while the model runs:

 On the server, once                     On the device, every token
 weights ─▶ fix weight scales            token ─▶ measure its min/max ─▶ scale ─▶ INT8

              one fixed scale (range ±50)          per-token scale
 Token A ±2   |·········█·········|  ~10 steps     |█████████████████|  all steps
 Token B ±50  |█████████████████|    all steps     |█████████████████|  all steps

Setting (Bonus recipe)	Meaning
weight CHANNEL + MINMAX	One scale per output channel of each Linear
activation TOKEN + MINMAX	Range measured per token at run time, so MINMAX is the only option
prefill / decode	The LLM is exported as two graphs, one that reads the whole prompt and one that generates one token at a time. Each is quantized separately

In the report, the last row shows the result of quantization. For WideResNet-50 (Step 7), Top-1 goes from 73.98 to 71.43 with the default settings. For the LLM (Bonus), the PTE file shrinks from 2522 MB to 1538 MB with the tuned recipe.

Step 6. Run It

np run --profile-method mlia --wait

• --profile-method mlia tells the server to estimate latency and memory with Arm ML Inference Advisor (MLIA) instead of measuring on a physical board. There is no device farm behind today's session, so always add it.
• --wait keeps the command open and shows progress until the run finishes.

You should see

[INFO ] [CLI] Submitting run (will wait for completion)
[INFO ] [CLI] Logs appear as each stage starts and ends. A quiet screen means the run is still queued or a stage is in progress.
[INFO ] [Workflow] Downloading input model started
...
════════════ [1/4] Graph Optimize ════════════
[INFO ] [Workflow] Graph Optimize started
...
[INFO ] [Workflow] Graph Optimize completed (3m 33s)
════════════ [2/4] Graph Quantize ════════════
...
[INFO ] [Workflow] Graph Quantize completed (6m 17s)
════════════ [3/4] Evaluate ════════════
...
[INFO ] [Workflow] Evaluate completed (50.4s)
════════════ [4/4] Profile ════════════
...
[INFO ] [Workflow] Run completed (16m 37s)
[DONE ] [CLI] Run completed successfully. (...)

[ Result Summary ]
 Experiment  <experiment_id>
 Status      success
Measurements
  Evaluate  base✓  01_graph_optimize✓  02_graph_quantize✓
  Profile   base✓  01_graph_optimize✓  02_graph_quantize✓  (estimated)

Artifacts
  NPIR      base✓  01_graph_optimize✓  02_graph_quantize✓

If the server is busy, Still submitted... 31s lines appear first while your run waits in the queue.

You may see some [WARN ] lines during the run. They are expected and the run still succeeds:

Warning contains	Why
is not in the process-kit catalog	The preset model uses its registered preprocessing values
PTE lowering failed ... zero delegated subgraphs	Ethos-U runs only quantized models, so no PTE is built for the FP32 model and the Graph Optimize output. They show n/a in the report
Per-file NPIR upload failed; falling back to the tar path	The server falls back to a single archive upload
SNR pad||pad: ... sample score(s) were non-finite	For a few images the two outputs are identical, so their SNR is infinite and left out of the average. The count varies by run, and the line may not appear at all

One run takes about 20 minutes, plus any time spent in the queue. Most of it goes to Graph Quantize (calibration on 98 images) and to latency and memory estimation (Profile). While you wait, ask a tutor a question.

Tip: Closed the terminal or pressed Ctrl+C?

The run keeps going on the Arm-hosted server; only the progress display stops. Check it from the experiment folder with:

np status

The experiment status moves from submitted to running to success. The two jobs finish first; accuracy evaluation (Evaluate) and latency estimation (Profile) run after them:

Experiment: experiment_00  Status: running
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Job               ┃ Type           ┃ Status  ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ 01_graph_optimize │ graph_optimize │ success │
│ 02_graph_quantize │ graph_quantize │ success │
└───────────────────┴────────────────┴─────────┘
Measurements
  Evaluate  running
  Profile   pending

The run is done when the first line reads Status: success and Measurements shows ✓ marks:

Experiment: experiment_00  Status: success
...
Measurements
  Evaluate  base✓  01_graph_optimize✓  02_graph_quantize✓
  Profile   base✓  01_graph_optimize✓  02_graph_quantize✓  (estimated)

Step 7. Read Your Results

np report

You should see one row per model version:

┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Model             ┃ Type           ┃ Status  ┃ Memory(MB) ┃ Latency(ms) ┃  MACs ┃ Params ┃ SNR (vs input) ┃ Top-1 Acc ┃ PTE Size(MB) ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ input_model       │ -              │ -       │       2.01 │      477.14 │ 11.4G │  69.0M │ -              │     73.98 │ n/a          │
│ 01_graph_optimize │ graph_optimize │ success │       2.01 │      477.14 │ 11.4G │  68.9M │ 199.11         │     73.98 │ n/a          │
│ 02_graph_quantize │ graph_quantize │ success │       2.01 │      482.76 │ 11.4G │  69.0M │ 46.53          │     71.43 │ 64.00        │
└───────────────────┴────────────────┴─────────┴────────────┴─────────────┴───────┴────────┴────────────────┴───────────┴──────────────┘




n/a  Lowering is not applicable: the backend delegates no operator of this model (Ethos-U requires a quantized model).
Size PTE file size after lowering; (%) vs input_model.

Your numbers may differ slightly. SNR and Top-1 can change a little from run to run.

How to read it: compare the first row (input_model, the original) with the last row (02_graph_quantize, the optimized model). The middle row shows that Graph Optimize alone keeps the model's accuracy: the same Top-1, and an SNR of about 200 means its output is practically identical to the original.

With the default settings the model is now INT8 and runs on Ethos-U85 (it gets a PTE file), but latency does not improve yet (477 → 483 ms) and Top-1 drops from 73.98 to 71.43. In Step 9 you rerun it with the tuned recipe and see the difference.

Column	What it means
Latency (ms)	Time for one inference on Alif E8, estimated by MLIA (not measured on a board). Lower is faster
Memory (MB)	Memory the model needs on the target. Lower is better
Top-1 Acc	Share of the 162 evaluation images classified correctly (%). Higher is better
PTE Size (MB)	Size of the ExecuTorch file for the board. n/a on rows Ethos-U can't run before quantization
SNR	How closely the output matches the original model's output. Higher means closer. input_model is the reference, so it shows -. Above about 100 dB the outputs are practically identical; the exact value can differ between servers

Sort the table by latency, slowest first:

np report --sort latency:desc

input_model always stays in the first row as the baseline; only the rows below it are sorted. Without :desc the order is fastest first.

Step 8. Look Inside the Graph with NetsPresso Probe

Probe opens the model graph in your browser, layer by layer. First set up a browser session:

np visualize auth

It prints a link and a token. Open the link within 3 minutes; it expires after that. If the link does not open the session, paste the token into the auth field on the Probe page instead. The browser session then lasts about an hour.

Then generate the graph link:

np visualize

Open the URL it prints. The [Tool] Using --tool probe (default) notice above the URL is normal.

What to look for

• How the graph changed after optimization: operators fused, quantize nodes added
• Added nodes are highlighted in green, removed nodes in red

Latency and memory are measured for the whole model, not per layer. Read them in np report.

Link expired? Run np visualize auth again.

Step 9. Run Again with the Tuned Recipe

Your first run used the default settings. Now run the same model again with the recipe we tuned for Alif E8, in a new experiment, so you can compare the two.

cd ..
np experiments create --steps go,gq
cd experiment_01

The recipe is two files:

• run.yaml: what runs. The steps, the quantization scheme, and which models are profiled and evaluated
• run.detail.yaml: how each step runs. Graph optimization rules, quantization granularity and calibration method, and the Ethos-U85 memory settings for profiling

In Step 5 you had no run.detail.yaml, so the server used its defaults. This time you create the file, see the defaults, and change two of them.

1) Replace run.yaml. It profiles only the original and the final model, which keeps the run short:

cat > run.yaml <<'EOF'
version: '1.0'
steps:
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8
    automatic_mixed_precision: null
profile:
- INPUT_MODEL
- OUTPUT_MODEL
evaluate:
- model: INPUT_MODEL
  metric:
  - TOP1_ACCURACY
- model: OUTPUT_MODEL
  metric:
  - TOP1_ACCURACY
EOF

2) Generate run.detail.yaml with the default settings:

np experiments detail

You should see

[DONE ] [CLI] Generated: run.detail.yaml

np experiments detail writes every setting the server would use for this run.yaml, with the allowed values in comments. Run it again after you change the steps in run.yaml, because the file lists settings per step.

3) Change two settings. Open run.detail.yaml (vi run.detail.yaml, nano run.detail.yaml, or your editor) and find the graph_quantize[0]: block near the bottom. It looks like this:

  graph_quantize[0]:
    weight:
      granularity: CHANNEL  # Available: CHANNEL, TENSOR
      calibration_method: MINMAX  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE
    activation:
      granularity: TENSOR  # Available: TENSOR
      calibration_method: PERCENTILE  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE
      percentile: 99.9

Change it to this. Two values change and the percentile line is deleted:

  graph_quantize[0]:
    weight:
      granularity: TENSOR  # Available: CHANNEL, TENSOR
      calibration_method: MINMAX  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE
    activation:
      granularity: TENSOR  # Available: TENSOR
      calibration_method: HISTOGRAM  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE

Setting	Default	Recipe	Meaning
weight.granularity	CHANNEL	TENSOR	One INT8 scale per weight tensor instead of one per output channel
activation.calibration_method	PERCENTILE (99.9)	HISTOGRAM	Activation ranges picked from a histogram of the 98 calibration images

Prefer one command instead of an editor?

This makes the same three changes (works on macOS and Linux):

sed -i.bak \
    -e 's/granularity: CHANNEL /granularity: TENSOR /' \
    -e 's/calibration_method: PERCENTILE /calibration_method: HISTOGRAM /' \
    -e '/percentile: 99.9/d' \
    run.detail.yaml

Full recipe run.detail.yaml (for reference)

If your edit went wrong, you can overwrite the file with the complete recipe. Copy from cat down to the last EOF:

cat > run.detail.yaml <<'EOF'
detail:
  batch_size: 1  # Applied to graph_quantize
  gpu_device: cuda:0  # Available: cpu, cuda:0
  graph_optimize[0]:
    pattern_handlers:
    - PatternHandlerCombineConvBN
    - PatternHandlerCombineConvBNHardtanh
    - PatternHandlerCombineConvBNRelu
    - PatternHandlerCombineConvHardtanh
    - PatternHandlerCombineConvRelu
    - PatternHandlerCombineConvTransposeBN
    - PatternHandlerCombineConvTransposeBNRelu
    - PatternHandlerCombineConvTransposeRelu
    - PatternHandlerCombineLinearHardtanh
    - PatternHandlerCombineLinearRelu
    - PatternHandlerCastInt64ConstOpsToInt32
    - PatternHandlerCastInt64InputsToInt32
    - PatternHandlerRestoreInt64RequiredArgs
    - PatternHandlerControlFlowConstInline
    - PatternHandlerConvertMinMax
    - PatternHandlerConvertSqueezeDimsToSqueezeDim
    - PatternHandlerFoldArangeToConstant
    - PatternHandlerFoldConstantPermute
    - PatternHandlerFoldTensorCreationOps
    - PatternHandlerFoldInplaceFillOnConstant
    - PatternHandlerDecomposeAdaptiveAvgPool2d
    - PatternHandlerDecomposeAddmm
    - PatternHandlerDecomposeAddSubAlpha
    - PatternHandlerDecomposeAsStridedCopy
    - PatternHandlerDecomposeAvgPool2d
    - PatternHandlerDecomposeCosineSimilarity
    - PatternHandlerDecomposeDiv
    - PatternHandlerDecomposeDivTensorMode
    - PatternHandlerDecomposeEmbedding
    - PatternHandlerDecomposeFloorDivide
    - PatternHandlerDecomposeGlu
    - PatternHandlerDecomposeIndexCopy
    - PatternHandlerDecomposeGroupNorm
    - PatternHandlerDecomposeLayerNorm
    - PatternHandlerDecomposeLeakyRelu
    - PatternHandlerDecomposeLinearVectorNorm
    - PatternHandlerDecomposeLogit
    - PatternHandlerDecomposeMaskedFill
    - PatternHandlerDecomposeMatmul
    - PatternHandlerDecomposeMaxDim
    - PatternHandlerDecomposeMeanDim
    - PatternHandlerDecomposeNotEqual
    - PatternHandlerDecomposePReluToReluMath
    - PatternHandlerDecomposeRemainder
    - PatternHandlerDecomposeRound
    - PatternHandlerDecomposeRoll
    - PatternHandlerDecomposeScaledDotProductAttention
    - PatternHandlerDecomposeSelectScatter
    - PatternHandlerDecomposeSign
    - PatternHandlerDecomposeSilu
    - PatternHandlerDecomposeSliceScatter
    - PatternHandlerDecomposeSoftmax
    - PatternHandlerDecomposeSqrt
    - PatternHandlerDecomposeStridedSlice
    - PatternHandlerDecomposeTril
    - PatternHandlerDecomposeVar
    - PatternHandlerDecomposeWhereCausalMask
    - PatternHandlerDecomposeWhereScalarOther
    - PatternHandlerDecomposeWhereSoftmaxNanCleanup
    - PatternHandlerPromoteBoolOperands
    - PatternHandlerNormalizeWhileInitialArgs
    - PatternHandlerReplaceBmmWithLinear
    - PatternHandlerDecomposeBmm
    - PatternHandlerFoldInverseReshapePair
    - PatternHandlerSplitQKV
    - PatternHandlerReplaceConv3dKd1WithConv2d
    - PatternHandlerReplaceInfAndLimitValues
    - PatternHandlerReplaceScalarWithTensorByProfile
    - PatternHandlerReplaceTensorOpScalarLiteralWithTensor
    - PatternHandlerCastInt64OutputOpsToInt32
    - PatternHandlerAdjustAdd
    - PatternHandlerAdjustConstantMathInputRank
    - PatternHandlerCanonicalizeConvolutionDefault
    - PatternHandlerCastBoolToInt8
    - PatternHandlerCommuteBNBeforeJoinOp
    - PatternHandlerCommuteBNBeforeLinearOp
    - PatternHandlerComposeErfFormulaIntoGelu
    - PatternHandlerComposeTanhFormulaIntoGelu
    - PatternHandlerFuseAffineInputToLinear
    - PatternHandlerFuseBNToConv
    - PatternHandlerFuseConsecutiveBatchNorm
    - PatternHandlerFuseConsecutiveCast
    - PatternHandlerFuseConsecutiveConcat
    - PatternHandlerFuseConsecutiveIdempotent
    - PatternHandlerFuseConsecutiveLogSoftmax
    - PatternHandlerFuseConsecutiveReshapeTranspose
    - PatternHandlerFuseConsecutiveSlice
    - PatternHandlerFuseConsecutiveUnsqueeze
    - PatternHandlerFuseConstantPath
    - PatternHandlerFuseInputScaleToConv
    - PatternHandlerFuseMathToConv
    - PatternHandlerFuseMathToLinear
    - PatternHandlerFusePadToConv
    - PatternHandlerFusePadToPool
    - PatternHandlerFuseReluPostfixToClamp
    - PatternHandlerFuseReluPrefixToClamp
    - PatternHandlerFuseMultiBranchReshapeTranspose
    - PatternHandlerFuseSplitConvConcat
    - PatternHandlerFuseUnsqueezeToReduce
    - PatternHandlerRemoveGraphAssertsPass
    - PatternHandlerRemoveBatchNormTracking
    - PatternHandlerRemoveBroadcastBeforeMath
    - PatternHandlerRemoveCommonSubexpression
    - PatternHandlerRemoveDeadend
    - PatternHandlerRemoveDisabledAutocastWrapper
    - PatternHandlerRemoveDropout
    - PatternHandlerRemoveMonotoneBeforeRanking
    - PatternHandlerRemoveIdentityPermutePair
    - PatternHandlerRemoveNoOpCast
    - PatternHandlerRemoveNoOpConcat
    - PatternHandlerRemoveNoOpExpand
    - PatternHandlerRemoveNoOpFlatten
    - PatternHandlerRemoveNoOpMath
    - PatternHandlerRemoveNoOpPad
    - PatternHandlerRemoveNoOpReshape
    - PatternHandlerRemoveNoOpSlice
    - PatternHandlerRemoveNoOpSplit
    - PatternHandlerRemoveNoOpSqueeze
    - PatternHandlerRemoveNoOpTranspose
    - PatternHandlerRemovePositive
    - PatternHandlerReplaceAddmmWithLinear
    - PatternHandlerReplaceBatchNormWithConv
    - PatternHandlerReplaceEinsumWithMatmul
    - PatternHandlerReplaceGatherWithConv
    - PatternHandlerReplaceGatherWithSliceReshape
    - PatternHandlerReplaceHwStride2SliceWithConv
    - PatternHandlerReplaceInplaceAdd
    - PatternHandlerReplaceInplaceDiv
    - PatternHandlerReplaceInplaceHardsigmoid
    - PatternHandlerReplaceInplaceHardswish
    - PatternHandlerReplaceInplaceIndexPut
    - PatternHandlerReplaceInplaceLeakyRelu
    - PatternHandlerReplaceInplaceMul
    - PatternHandlerReplaceInplaceRelu
    - PatternHandlerReplaceInplaceSub
    - PatternHandlerReplaceInplaceSilu
    - PatternHandlerReplaceNegWithConv
    - PatternHandlerReplaceTypeAsWithCast
  graph_quantize[0]:
    weight:
      granularity: TENSOR  # Available: TENSOR, CHANNEL
      calibration_method: MINMAX  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE
    activation:
      granularity: TENSOR  # Available: TENSOR
      calibration_method: HISTOGRAM  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE
    parameters:
      calibration:
        num_samples: -1
    # -- Manual Mixed Precision (optional) --
    # Override precision of specific modules/operators on top of base scheme
    # Can be used together with automatic_mixed_precision (Manual MP targets are excluded from AMP)
    # Priority: Manual MP > AMP > Base scheme
    #
    # --- Example ---
    # mixed_precision:
    #   by_operator:                    # Override by operator type
    #     LINEAR:
    #       weight:
    #         dtype: INT4
    #         granularity: GROUP
    #         group_size: 128
    #     CONV: null                    # Skip quantization for this operator type
    #   by_module:                      # Override by NPIR module ID
    #     "42":
    #       weight:
    #         dtype: INT4               # Override only weight from base scheme
    #     "57": null                    # Skip quantization for this module
  profile[0]:
    num_executions: 30
    timeout: 60
    enable_etdump: false
    memory_mode: Shared_Sram
    system_config: Ethos_U85_SRAM_OSPI
  profile[1]:
    num_executions: 30
    timeout: 60
    enable_etdump: false
    memory_mode: Shared_Sram
    system_config: Ethos_U85_SRAM_OSPI
EOF

4) Validate:

np validate

You should see

[DONE ] [CLI] Validation Passed. No issues found.
                         Warnings
┏━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Code   ┃ Level ┃ Location                         ┃ Message                                             ┃
┡━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ NP-301 │ WARN  │ run.detail.yaml > detail.profile │ ETDump is disabled (enable_etdump: false) for       │
│        │       │                                  │ Ethos-U target. Layerwise tensor-level score will   │
│        │       │                                  │ not be generated for this run.                      │
└────────┴───────┴──────────────────────────────────┴─────────────────────────────────────────────────────┘
╭──────────────── Execution Plan ────────────────╮
│ Steps:                                          │
│   [1/2] graph_optimize                          │
│   [2/2] graph_quantize — scheme: W8-A8          │
│ Profile: INPUT_MODEL, OUTPUT_MODEL              │
│ Evaluate:                                       │
│   INPUT_MODEL (top1_accuracy)                   │
│   OUTPUT_MODEL (top1_accuracy)                  │
│ Total: 6 jobs                                   │
╰─────────────────────────────────────────────────╯

The NP-301 warning is expected. ETDump (per-layer execution trace) is off by default to keep the run short. Continue.

During this run you may also see run.detail.yaml profile options are not supported ... runner_timeout. One profiling option in the file does not apply to the MLIA estimate; it is ignored and the run still succeeds.

5) Run it and wait about 15 minutes, then read the report:

np run --profile-method mlia --wait
np report

You should see

┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Model             ┃ Type           ┃ Status  ┃ Memory(MB) ┃ Latency(ms) ┃  MACs ┃ Params ┃ Top-1 Acc ┃ PTE Size(MB) ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ input_model       │ -              │ -       │       2.01 │      477.14 │ 11.4G │  69.0M │     73.98 │ n/a          │
│ 01_graph_optimize │ graph_optimize │ success │          - │           - │     - │      - │         - │ n/a          │
│ 02_graph_quantize │ graph_quantize │ success │       1.96 │      350.24 │ 11.4G │  68.9M │     73.47 │ 42.71        │
└───────────────────┴────────────────┴─────────┴────────────┴─────────────┴───────┴────────┴───────────┴──────────────┘

01_graph_optimize shows - because the recipe measures only the original and the final model. The SNR column is not shown because the recipe does not measure SNR.

Compare it with your first run:

	Default (experiment_00)	Tuned recipe (experiment_01)
Latency (ms)	477.14 → 482.76	477.14 → 350.24 (−27%)
Memory (MB)	2.01 → 2.01	2.01 → 1.96
Top-1 Acc (%)	73.98 → 71.43	73.98 → 73.47
PTE Size (MB)	64.00	42.71

To see both experiments in one table, run np report from the project folder:

cd ..
np report

Tip: Want to try one more change?

Copy the recipe into a third experiment and turn on automatic mixed precision, which keeps sensitive layers at higher precision. --copy-config copies both run.yaml and run.detail.yaml:

np experiments create --copy-config experiment_01
cd experiment_02

In run.yaml, change automatic_mixed_precision: null to automatic_mixed_precision: GREEDY, then run np validate (the plan shows scheme: W8-A8, amp: GREEDY) and np run --profile-method mlia --wait.

Caution: Keep scheme: W8-A8 on Alif E8

With W8-A16, WideResNet-50 exceeds the 7 MB SRAM of Alif E8 and PTE generation fails. Ethos-U does not run dynamic schemes, so W8-A8_dynamic and W4-A8_dynamic fail validation.

Step 10. Take the Model Home

From your best experiment folder:

cd experiment_01   # the tuned recipe
np export

You should see

[DONE ] [CLI] Export completed.

[ Result Summary ]
 Level         deploy
 Extracted to  /.../wrn50-alif/experiment_01

[ Result ]
  - model.pte

model.pte (about 43 MB) is saved in the experiment folder. Exporting experiment_00 instead gives the default-settings model, about 64 MB.

model.pte is an ExecuTorch program for Alif E8. On the board it is built into the firmware image together with the ExecuTorch runtime.

Want the configuration too? np export --level config. The measurement results as well: np export --level result.

Bonus: LLM on AWS Graviton4

The presenter ran this LLM job live at the start of the session. Finished early? You can run the same job yourself. It uses Advanced Quantize (aq, AWQ) before Graph Optimize and Graph Quantize. Each run takes about an hour, so you may not see the result before the session ends; if you only have time for one, skip to the tuned recipe. Your job keeps running on the server; check it later with np status. For what AWQ does, open "How the optimization steps work" at the end of Step 5.

Item	Value
Model	qwen2_5_0_5b_instruct_hf_pytorch (Qwen2.5-0.5B-Instruct, preset)
Device	graviton4 (AWS Graviton4, Neoverse V2)
Runtime / Backend	executorch / xnnpack
Calibration dataset	common_pile_calib_seed42_n256 (Common Pile, 256 documents)
Evaluation dataset	OpenBookQA (500 questions, 0-shot)
Metric	ACCURACY

cd ~/arm-create/workspace        # your workspace folder
np projects create \
    --name qwen-graviton4 \
    --model qwen2_5_0_5b_instruct_hf_pytorch \
    --device graviton4 \
    --runtime executorch \
    --backend xnnpack \
    --calibration-dataset common_pile_calib_seed42_n256
cd qwen-graviton4
np experiments create --steps aq,go,gq
cd experiment_00

Open run.yaml and add OPENBOOKQA to the metric list of INPUT_MODEL and OUTPUT_MODEL. Leave everything else as generated (the default scheme is W4-A8_dynamic). Then:

np validate
np run --profile-method mlia

Submit LLM runs without --wait. Accuracy evaluation of an LLM prints many [WARN ] [np_evaluator] lines, which are expected but make the progress display hard to read. Check progress with np status; when the first line reads Status: success, run np report:

Model	OpenBookQA	PTE Size (MB)	Delegated (XNNPACK)
input_model	0.26	-	-
01_advanced_quantize	-	2522.22	38.0%
03_graph_quantize	0.27	1102.08	54.5%

The default scheme W4-A8_dynamic stores weights in 4 bits, so its PTE file is smaller than the tuned recipe's below. With the default settings OpenBookQA is scored on 100 questions. The tuned recipe below scores all 500, so the two runs' accuracy numbers are not directly comparable.

Run again with the tuned recipe

The presenter's demo used the recipe below. To reproduce it, create a new experiment and replace both files, as in Step 9:

cd ..
np experiments create --steps aq,go,gq
cd experiment_01

1) run.yaml

cat > run.yaml <<'EOF'
version: '1.0'
steps:
- advanced_quantize:
  - algorithm: AWQ
    scheme: W8-A8_dynamic
    components:
      prefill:
        algorithm: AWQ
        scheme: W8-A8_dynamic
      decode:
        algorithm: AWQ
        scheme: W8-A8_dynamic
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8_dynamic
    automatic_mixed_precision: null
    components:
      prefill:
        scheme: W8-A8_dynamic
        automatic_mixed_precision: null
      decode:
        scheme: W8-A8_dynamic
        automatic_mixed_precision: null
profile:
- INPUT_MODEL
- OUTPUT_MODEL
evaluate:
- model: INPUT_MODEL
  metric:
  - OPENBOOKQA
- model: OUTPUT_MODEL
  metric:
  - OPENBOOKQA
EOF

2) run.detail.yaml

np experiments detail would generate this file with the default settings, as in Step 9. The recipe changes more than a few values here (AWQ calibration samples 128 → 30, OpenBookQA 100 → 500 questions, and separate prefill / decode settings), so write the whole file instead. Copy from cat down to the last EOF:

cat > run.detail.yaml <<'EOF'
detail:
  batch_size: 1  # Applied to advanced_quantize, graph_quantize, evaluate
  gpu_device: cuda:0  # Available: cpu, cuda:0
  advanced_quantize[0]:
    weight:
      granularity: CHANNEL  # Available: CHANNEL
    activation:
      granularity: TOKEN  # Available: TOKEN, TENSOR
    parameters:
      calibration:
        num_samples: 30
      clip: true  # Available: true, false
    components:
      prefill:
        weight:
          granularity: CHANNEL
        activation:
          granularity: TOKEN
        parameters:
          calibration:
            num_samples: 30
          clip: true
      decode:
        weight:
          granularity: CHANNEL
        activation:
          granularity: TOKEN
        parameters:
          calibration:
            num_samples: 30
          clip: true
    # -- Manual Mixed Precision (optional) --
    # Override precision of specific layers on top of base scheme
    # Priority: Manual MP > Base scheme
    #
    # --- Example ---
    # mixed_precision:
    #   by_name:                          # Override by regex pattern
    #     "model.layers.0.*":
    #       weight:
    #         dtype: INT4
    #     "model.lm_head": null           # Skip quantization for this layer
  graph_optimize[0]:
    pattern_handlers:
    - PatternHandlerAdjustAdd
    - PatternHandlerAdjustConstantMathInputRank
    - PatternHandlerAdjustNonContiguousConstantMathInput
    - PatternHandlerCanonicalizeConvolutionDefault
    - PatternHandlerCombineAddRelu
    - PatternHandlerCombineConvBN
    - PatternHandlerCombineConvBNRelu
    - PatternHandlerCombineConvRelu
    - PatternHandlerCombineConvTransposeBN
    - PatternHandlerCombineConvTransposeBNRelu
    - PatternHandlerCombineConvTransposeRelu
    - PatternHandlerCombineLinearRelu
    - PatternHandlerCombineMulRelu
    - PatternHandlerCombineSDPA
    - PatternHandlerCommuteBNBeforeJoinOp
    - PatternHandlerCommuteBNBeforeLinearOp
    - PatternHandlerComposeErfFormulaIntoGelu
    - PatternHandlerComposeTanhFormulaIntoGelu
    - PatternHandlerDecomposeAvgPool2d
    - PatternHandlerDecomposeLogit
    - PatternHandlerDecomposeMatmul
    - PatternHandlerDecomposeMaxDimKeepdimFalse
    - PatternHandlerDecomposePReluToReluMath
    - PatternHandlerDecomposeScaledDotProductAttention
    - PatternHandlerDecomposeStridedSlice
    - PatternHandlerDecomposeTril
    - PatternHandlerDecomposeWhereScalarOther
    - PatternHandlerFoldConstantBoolMask
    - PatternHandlerFoldConstantPermute
    - PatternHandlerFuseAffineInputToLinear
    - PatternHandlerFuseAsymmetricPadToConv
    - PatternHandlerFuseBNToConv
    - PatternHandlerFuseConsecutiveBatchNorm
    - PatternHandlerFuseConsecutiveCast
    - PatternHandlerFuseConsecutiveConcat
    - PatternHandlerFuseConsecutiveIdempotent
    - PatternHandlerFuseConsecutiveLogSoftmax
    - PatternHandlerFuseConsecutiveReshapeTranspose
    - PatternHandlerFuseConsecutiveSlice
    - PatternHandlerFuseConsecutiveUnsqueeze
    - PatternHandlerFoldArangeToConstant
    - PatternHandlerFoldTensorCreationOps
    - PatternHandlerFoldInplaceFillOnConstant
    - PatternHandlerFuseConstantPath
    - PatternHandlerFuseInputScaleToConv
    - PatternHandlerFuseMathToConv
    - PatternHandlerFuseMathToLinear
    - PatternHandlerFusePadToConv
    - PatternHandlerFusePadToPool
    - PatternHandlerFuseReluPostfixToClamp
    - PatternHandlerFuseReluPrefixToClamp
    - PatternHandlerFuseMultiBranchReshapeTranspose
    - PatternHandlerFuseQKV
    - PatternHandlerFuseSplitConvConcat
    - PatternHandlerFuseUnsqueezeToReduce
    - PatternHandlerNormalizeMeanDimLastKeepdim
    - PatternHandlerNormalizeSpatialMeanKeepdimFalse
    - PatternHandlerRemoveGraphAssertsPass
    - PatternHandlerRemoveBatchNormTracking
    - PatternHandlerRemoveBroadcastBeforeMath
    - PatternHandlerRemoveCommonSubexpression
    - PatternHandlerRemoveDeadend
    - PatternHandlerRemoveDisabledAutocastWrapper
    - PatternHandlerRemoveDropout
    - PatternHandlerRemoveMonotoneBeforeRanking
    - PatternHandlerRemoveNoOpCast
    - PatternHandlerRemoveNoOpConcat
    - PatternHandlerRemoveNoOpExpand
    - PatternHandlerRemoveNoOpFlatten
    - PatternHandlerRemoveNoOpMath
    - PatternHandlerRemoveNoOpPad
    - PatternHandlerRemoveNoOpReshape
    - PatternHandlerRemoveNoOpSlice
    - PatternHandlerRemoveNoOpSplit
    - PatternHandlerRemoveNoOpSqueeze
    - PatternHandlerRemoveNoOpTranspose
    - PatternHandlerRemovePositive
    - PatternHandlerReplaceAddmmWithLinear
    - PatternHandlerReplaceBatchNormWithConv
    - PatternHandlerReplaceBitwiseRightShiftWithDiv
    - PatternHandlerReplaceConv3dFullWindowWithLinear
    - PatternHandlerReplaceEinsumWithMatmul
    - PatternHandlerReplaceGatherWithConv
    - PatternHandlerReplaceGatherWithSliceReshape
    - PatternHandlerReplaceInplaceAdd
    - PatternHandlerReplaceInplaceHardswish
    - PatternHandlerReplaceInplaceIndexPut
    - PatternHandlerReplaceInplaceLeakyRelu
    - PatternHandlerReplaceInplaceMul
    - PatternHandlerReplaceInplaceRelu
    - PatternHandlerReplaceInplaceSilu
    - PatternHandlerReplaceMatmul2dWithLinear
    - PatternHandlerReplaceMatmul3dWithLinear
    - PatternHandlerReplaceMatmulWithBmm
    - PatternHandlerReplaceMatmulWithLinear
    - PatternHandlerReplaceSqueezeWithReshape
    - PatternHandlerReplaceTypeAsWithCast
  graph_quantize[0]:
    weight:
      granularity: CHANNEL  # Available: CHANNEL
      calibration_method: MINMAX  # Available: MINMAX, HISTOGRAM, MOVING_AVERAGE_MINMAX, PERCENTILE
    activation:
      granularity: TOKEN  # Available: TOKEN, TENSOR
      calibration_method: MINMAX  # Available: MINMAX
    components:
      prefill:
        weight:
          granularity: CHANNEL
          calibration_method: MINMAX
        activation:
          granularity: TOKEN
          calibration_method: MINMAX
      decode:
        weight:
          granularity: CHANNEL
          calibration_method: MINMAX
        activation:
          granularity: TOKEN
          calibration_method: MINMAX
    # -- Manual Mixed Precision (optional) --
    # Override precision of specific modules/operators on top of base scheme
    # Can be used together with automatic_mixed_precision (Manual MP targets are excluded from AMP)
    # Priority: Manual MP > AMP > Base scheme
    #
    # --- Example ---
    # mixed_precision:
    #   by_operator:                    # Override by operator type
    #     LINEAR:
    #       weight:
    #         dtype: INT4
    #         granularity: GROUP
    #         group_size: 128
    #     CONV: null                    # Skip quantization for this operator type
    #   by_module:                      # Override by NPIR module ID
    #     "42":
    #       weight:
    #         dtype: INT4               # Override only weight from base scheme
    #     "57": null                    # Skip quantization for this module
  profile[0]:
    num_executions: 30
    timeout: 60
  profile[1]:
    num_executions: 30
    timeout: 60
  evaluate[0]:
    metrics:
    - OPENBOOKQA:
        limit: 500  # Available: 1 ~ 500
        num_fewshot: 0  # Available: 0 ~ (lm-eval task default is 0)
        apply_chat_template: false  # Available: true, false
  evaluate[1]:
    metrics:
    - OPENBOOKQA:
        limit: 500  # Available: 1 ~ 500
        num_fewshot: 0  # Available: 0 ~ (lm-eval task default is 0)
        apply_chat_template: false  # Available: true, false
EOF

Then:

np validate
np run --profile-method mlia

You should see np validate pass with no warnings and plan 7 jobs: advanced_quantize — algorithm: AWQ, scheme: W8-A8_dynamic, graph_optimize, graph_quantize — scheme: W8-A8_dynamic.

Check progress with np status. When it finishes, run np report. Compare PTE size and the Delegated (XNNPACK) column, the share of operators that run on the XNNPACK CPU backend:

Model	PTE Size (MB)	Delegated (XNNPACK)
01_advanced_quantize	2522.22	38.0%
03_graph_quantize	1537.94	54.5%

The OpenBookQA column rounds to two decimals, so it can show the same value for both models. See both OpenBookQA scores with:

np report --output

   Metric Detail: INPUT_MODEL
          (OPENBOOKQA)
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ Key                 ┃  Value ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ openbookqa_acc      │ 0.2420 │
│ openbookqa_acc_norm │ 0.3480 │
└─────────────────────┴────────┘
Metric Detail: 03_graph_quantize
...

The scores of the quantized model vary by about a point from run to run.

Latency and memory show as n/a: MLIA estimates them for Ethos-U only, and there is no device farm behind today's session.

If Something Goes Wrong

What you see	What to do
np: command not found	Activate the venv: source .venv/bin/activate from the arm-create folder (Git Bash: source .venv/Scripts/activate, PowerShell: .\.venv\Scripts\Activate.ps1)
PowerShell: Activate.ps1 cannot be loaded because running scripts is disabled	Run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned, then activate again
PowerShell: The token '&&' is not a valid statement separator	Run the commands one per line
pip install fails on Python version	Check python3 --version is 3.11 or later
pip: command not found	Activate the venv (source .venv/bin/activate), then install with python -m pip install … instead of pip install …
Login fails	Check the token was copied in one piece, then run np auth status
Model ... not found or Dataset ... not found	Check the spelling against Today's settings. Names are exact matches
Status stays submitted	The server is busy with other participants' jobs. Keep waiting, or check later with np status
Pressed Ctrl+C or closed the terminal during np run --profile-method mlia --wait	The run keeps going on the server. Check it with np status from the experiment folder
np visualize link doesn't open	Run np visualize auth again and open its link within 3 minutes
Validation fails after editing run.yaml	Check indentation (2 spaces, no tabs) and that the scheme is W8-A8
np validate shows warning NP-301 (ETDump) in Step 9	Expected. Continue
Step 9 latency stays around 480 ms	The two run.detail.yaml changes did not apply. Check with grep -n "CHANNEL |PERCENTILE " run.detail.yaml (should print nothing), fix the file in a new experiment and run again
Want to stop a job	np cancel from the experiment folder
Anything else	Raise your hand. A tutor will come to you

For a failed run, np status --logs prints the server log. Show it to a tutor.

Every Command on One Page

# 1 Install
python3 -m venv .venv && source .venv/bin/activate
python -m pip install arm-ai-model-optimizer

# 2 Log in
np health ready
np auth login <your-token>

# 3 Workspace
np workspace init && cd workspace

# 4 Project
np projects create --name wrn50-alif --model wideresnet50_cls_14_pt2_pytorch_static_fp32 \
  --device alif_e8_devkit --runtime executorch --backend ethos_u \
  --calibration-dataset openimages_filtered_calib_seed42_n98 \
  --evaluation-dataset openimages_filtered_eval_seed42_n162

# 5 Experiment
cd wrn50-alif && np experiments create --steps go,gq && cd experiment_00
#   (replace run.yaml, see Step 5)
np validate

# 6 Run
np run --profile-method mlia --wait

# 7 Report
np report
np report --sort latency:desc

# 8 Probe
np visualize auth
np visualize

# 9 Tuned recipe
cd .. && np experiments create --steps go,gq && cd experiment_01
#   (replace run.yaml, see Step 9)
np experiments detail
#   (edit run.detail.yaml: weight granularity TENSOR, activation calibration_method HISTOGRAM)
np validate
np run --profile-method mlia --wait
np report

# 10 Export
np export          # from experiment_01

After the Session

• Keep going: request Early Access to use the optimizer on Arm AI Portal: [EAP_LINK]
• Survey: 3 minutes, tell us which devices, models and features you need: Arm Create hands-on survey
• Full documentation: the general User Guide covers bringing your own model and data (BYOM / BYOD), other devices and every CLI option.

Your session token stops working after [TOKEN_EXPIRY]. Results you exported stay on your laptop.