Chapter 8: Best Practices & Case Studies

WIA-AI-008 Standard • Estimated reading time: 45 minutes

8.1 End-to-End Model Exchange Workflow

A robust model exchange workflow integrates all concepts from previous chapters into a cohesive pipeline that spans training, optimization, versioning, distribution, and deployment.

Complete Pipeline Example

#!/usr/bin/env python3
"""
Complete model exchange pipeline following WIA-AI-008 standard
弘益人間 - Benefit All Humanity
"""

import torch
import mlflow
import onnx
from datetime import datetime

class ModelExchangePipeline:
    def __init__(self, config):
        self.config = config
        mlflow.set_tracking_uri(config['mlflow_uri'])
        mlflow.set_experiment(config['experiment_name'])

    def train(self):
        """Phase 1: Train model"""
        print("📚 Phase 1: Training model...")

        with mlflow.start_run():
            # Train model
            model = self._train_model()

            # Log everything
            mlflow.log_params(self.config['hyperparameters'])
            mlflow.log_metrics(self.metrics)
            mlflow.log_artifact("training_logs.txt")

            # Save PyTorch model
            torch.save(model.state_dict(), "model.pth")

            return model

    def optimize(self, model):
        """Phase 2: Optimize model"""
        print("⚡ Phase 2: Optimizing model...")

        # Quantization
        quantized_model = torch.quantization.quantize_dynamic(
            model, {torch.nn.Linear}, dtype=torch.qint8
        )

        # Prune
        # ... pruning code ...

        # Validate accuracy
        accuracy_drop = self._validate_optimization(model, quantized_model)
        assert accuracy_drop < 0.02, "Accuracy degradation too high"

        return quantized_model

    def export(self, model):
        """Phase 3: Export to multiple formats"""
        print("📦 Phase 3: Exporting to multiple formats...")

        # Export to ONNX
        dummy_input = torch.randn(1, *self.config['input_shape'])
        torch.onnx.export(
            model, dummy_input, "model.onnx",
            opset_version=15, export_params=True,
            input_names=['input'], output_names=['output'],
            dynamic_axes={'input': {0: 'batch'}, 'output': {0: 'batch'}}
        )

        # Validate ONNX
        onnx_model = onnx.load("model.onnx")
        onnx.checker.check_model(onnx_model)

        # Export to TorchScript
        scripted_model = torch.jit.script(model)
        scripted_model.save("model.pt")

        print("✓ Exported to ONNX and TorchScript")

    def create_model_card(self):
        """Phase 4: Create comprehensive model card"""
        print("📝 Phase 4: Creating model card...")

        model_card = {
            'name': self.config['model_name'],
            'version': self.config['version'],
            'created_at': datetime.now().isoformat(),
            'framework': 'pytorch',
            'architecture': self.config['architecture'],
            'metrics': self.metrics,
            'hyperparameters': self.config['hyperparameters'],
            'intended_use': 'Image classification',
            'limitations': 'Limited to ImageNet classes',
            'ethical_considerations': 'May have biases from training data',
            'philosophy': '弘益人間 - Benefit All Humanity'
        }

        with open("model_card.json", "w") as f:
            json.dump(model_card, f, indent=2)

    def register(self):
        """Phase 5: Register to model registry"""
        print("🏛️ Phase 5: Registering to MLflow...")

        mlflow.pytorch.log_model(
            self.model,
            "model",
            registered_model_name=self.config['model_name']
        )

        # Transition to staging
        client = mlflow.tracking.MlflowClient()
        versions = client.search_model_versions(f"name='{self.config['model_name']}'")
        latest_version = versions[0].version

        client.transition_model_version_stage(
            name=self.config['model_name'],
            version=latest_version,
            stage="Staging"
        )

        print(f"✓ Registered version {latest_version}")

    def deploy(self):
        """Phase 6: Deploy model"""
        print("🚀 Phase 6: Deploying model...")

        # Create TorchServe archive
        os.system(f"""
            torch-model-archiver \\
                --model-name {self.config['model_name']} \\
                --version {self.config['version']} \\
                --serialized-file model.pt \\
                --handler image_classifier \\
                --export-path model-store/
        """)

        # Deploy to TorchServe
        # ... deployment code ...

        print("✓ Model deployed")

    def run_pipeline(self):
        """Execute complete pipeline"""
        print("=" * 60)
        print("WIA-AI-008 Model Exchange Pipeline")
        print("弘益人間 - Benefit All Humanity")
        print("=" * 60)

        model = self.train()
        model = self.optimize(model)
        self.export(model)
        self.create_model_card()
        self.register()
        self.deploy()

        print("\n✅ Pipeline completed successfully!")

# Usage
config = {
    'model_name': 'resnet18-classifier',
    'version': '1.0.0',
    'experiment_name': 'image-classification',
    'mlflow_uri': 'http://localhost:5000',
    'architecture': 'ResNet-18',
    'input_shape': [3, 224, 224],
    'hyperparameters': {
        'learning_rate': 0.001,
        'batch_size': 32,
        'epochs': 10
    }
}

pipeline = ModelExchangePipeline(config)
pipeline.run_pipeline()

8.2 Case Study 1: Hugging Face Model Sharing

from transformers import BertForSequenceClassification, BertTokenizer, TrainingArguments, Trainer
from huggingface_hub import HfApi, create_repo

# 1. Fine-tune model
model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

trainer = Trainer(model=model, args=training_args, train_dataset=train_dataset)
trainer.train()

# 2. Save model and tokenizer
model.save_pretrained("./sentiment-bert")
tokenizer.save_pretrained("./sentiment-bert")

# 3. Create comprehensive README (model card)
readme = """---
language: en
license: apache-2.0
tags:
- sentiment-analysis
- bert
- text-classification
datasets:
- imdb
metrics:
- accuracy: 0.934
- f1: 0.931
model-index:
- name: sentiment-bert
  results:
  - task:
      type: text-classification
      name: Sentiment Analysis
    dataset:
      name: IMDB
      type: imdb
    metrics:
    - type: accuracy
      value: 0.934
---

# Sentiment Analysis BERT

Fine-tuned BERT-base model for sentiment analysis on IMDB reviews.

## Model Details
- Base model: bert-base-uncased
- Task: Binary sentiment classification (positive/negative)
- Training data: IMDB dataset (50,000 reviews)
- Framework: PyTorch + Transformers

## Usage
```python
from transformers import pipeline

classifier = pipeline('sentiment-analysis', model='username/sentiment-bert')
result = classifier('This movie was amazing!')
# [{'label': 'POSITIVE', 'score': 0.9987}]
```

## Training Procedure
- Learning rate: 2e-5
- Batch size: 16
- Epochs: 3
- Optimizer: AdamW

## Evaluation Results
- Accuracy: 93.4%
- F1 Score: 93.1%
- Precision: 94.2%
- Recall: 92.0%

## Limitations
- English language only
- Trained on movie reviews, may not generalize to other domains
- May exhibit biases present in IMDB dataset

## Ethical Considerations
弘益人間 (Benefit All Humanity) - This model is intended for research and educational purposes.
Users should be aware of potential biases and use responsibly.

## Citation
```bibtex
@misc{sentiment-bert,
  author = {Research Team},
  title = {Fine-tuned BERT for Sentiment Analysis},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\\url{https://huggingface.co/username/sentiment-bert}}
}
```
"""

with open("./sentiment-bert/README.md", "w") as f:
    f.write(readme)

# 4. Create repository and upload
repo_id = "username/sentiment-bert"
create_repo(repo_id, exist_ok=True)

api = HfApi()
api.upload_folder(
    folder_path="./sentiment-bert",
    repo_id=repo_id,
    commit_message="Initial model upload"
)

print(f"✓ Model published: https://huggingface.co/{repo_id}")

8.3 Case Study 2: Cross-Platform Mobile Deployment

# PyTorch (Training) → ONNX → TFLite (Android)
#                            ↓
#                        CoreML (iOS)

# Step 1: Export PyTorch to ONNX
import torch
from models.yolo import YOLOv5

model = YOLOv5.from_pretrained('yolov5s')
model.eval()

dummy_input = torch.randn(1, 3, 640, 640)
torch.onnx.export(
    model, dummy_input, "yolov5s.onnx",
    opset_version=12, input_names=['images'],
    output_names=['output'], dynamic_axes={'images': {0: 'batch'}}
)

# Step 2: ONNX to CoreML (iOS)
import coremltools as ct
from onnx_coreml import convert

coreml_model = convert(model='yolov5s.onnx')
coreml_model.save('YOLOv5.mlmodel')

# Step 3: ONNX to TensorFlow to TFLite (Android)
from onnx_tf.backend import prepare
import tensorflow as tf

onnx_model = onnx.load("yolov5s.onnx")
tf_model = prepare(onnx_model)
tf_model.export_graph("yolov5_tf")

converter = tf.lite.TFLiteConverter.from_saved_model("yolov5_tf")
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

with open("yolov5s.tflite", "wb") as f:
    f.write(tflite_model)

# Step 4: Numerical validation
validate_cross_platform(pytorch_model, coreml_model, tflite_model)

8.4 Case Study 3: MLOps Pipeline at Scale

# .gitlab-ci.yml
stages:
  - train
  - test
  - optimize
  - register
  - deploy

train:
  stage: train
  script:
    - python train.py --config config.yaml
    - python validate.py --threshold 0.90
  artifacts:
    paths:
      - model.pth
      - metrics.json

test:
  stage: test
  script:
    - pytest tests/test_model.py
    - python benchmark.py --latency-threshold 100
  dependencies:
    - train

optimize:
  stage: optimize
  script:
    - python quantize.py --model model.pth
    - python export_onnx.py
    - python validate_onnx.py
  artifacts:
    paths:
      - model_quantized.pth
      - model.onnx

register:
  stage: register
  script:
    - python register_model.py --name $CI_COMMIT_TAG
  only:
    - tags

deploy:
  stage: deploy
  script:
    - python deploy_triton.py --version $CI_COMMIT_TAG
  when: manual
  only:
    - tags

8.5 Best Practices Checklist

Model Development

• ☑️ Use version control (Git) for code
• ☑️ Track experiments with MLflow/W&B
• ☑️ Document hyperparameters and configurations
• ☑️ Save training logs and metrics
• ☑️ Use reproducible random seeds

Model Export & Conversion

• ☑️ Export to framework-agnostic format (ONNX)
• ☑️ Validate numerical equivalence after conversion
• ☑️ Test on representative data samples
• ☑️ Document conversion steps and parameters
• ☑️ Include fallback for unsupported operations

Model Optimization

• ☑️ Benchmark before optimization (baseline)
• ☑️ Apply quantization with calibration data
• ☑️ Measure accuracy impact of each optimization
• ☑️ Profile inference performance
• ☑️ Document size/speed/accuracy trade-offs

Model Registry

• ☑️ Use semantic versioning (MAJOR.MINOR.PATCH)
• ☑️ Include comprehensive model cards
• ☑️ Tag models with metadata (task, dataset, metrics)
• ☑️ Implement access control and permissions
• ☑️ Maintain staging → production workflow

Model Deployment

• ☑️ Containerize serving infrastructure (Docker)
• ☑️ Implement health checks and readiness probes
• ☑️ Set up monitoring and alerting
• ☑️ Use load balancing and auto-scaling
• ☑️ Plan for rollback and disaster recovery

Production Monitoring

• ☑️ Track latency, throughput, error rates
• ☑️ Monitor model performance (accuracy drift)
• ☑️ Log predictions for debugging
• ☑️ Set up alerts for anomalies
• ☑️ Implement A/B testing infrastructure

8.6 Common Pitfalls and Solutions

8.7 Tools and Resources

Essential Tools

• ONNX: Framework-agnostic model format
• ONNX Runtime: High-performance inference engine
• Netron: Model visualization
• MLflow: Experiment tracking and model registry
• DVC: Data and model version control
• TorchServe: PyTorch model serving
• TensorFlow Serving: TensorFlow model serving
• NVIDIA Triton: Multi-framework serving
• Prometheus: Metrics and monitoring
• Grafana: Visualization and dashboards

Learning Resources

• WIA-AI-008 Specification: wia.org/standards/ai-008
• ONNX Tutorials: github.com/onnx/tutorials
• MLflow Documentation: mlflow.org/docs
• Model Cards Paper: arxiv.org/abs/1810.03993
• TorchServe GitHub: github.com/pytorch/serve

8.8 Future of Model Exchange

Emerging Trends

• Federated Learning: Training across distributed data without centralization
• Neural Architecture Search (NAS): Automated model design for target platforms
• Model Compression: Advanced techniques (knowledge distillation, pruning, quantization)
• Edge AI: On-device inference with TinyML
• Model Marketplaces: Commercial model exchange platforms
• AI Regulations: Compliance requirements for model documentation

The Role of WIA-AI-008

The WIA-AI-008 standard provides a foundation for these future developments, ensuring interoperability, transparency, and ethical deployment of AI models. As the field evolves, the standard will adapt to incorporate new best practices while maintaining its core philosophy of 弘益人間 (Benefit All Humanity).

8.9 Conclusion

Model exchange is no longer optional—it's essential for modern ML operations. By following the WIA-AI-008 standard and the best practices outlined in this book, teams can:

• Accelerate development through model sharing and reuse
• Deploy models across diverse platforms and frameworks
• Maintain high standards for reproducibility and documentation
• Build production-ready ML systems with confidence
• Contribute to the open AI ecosystem

The journey from training to production is complex, but with the right tools, processes, and standards, it becomes manageable and repeatable. We hope this book serves as a comprehensive guide for your model exchange needs.