Skip to content

Lab 0: Workshop Pre-Work

Welcome to the Output Drift workshop! This lab will guide you through setting up your environment and ensuring you have all the prerequisites installed.

Prerequisites

Before you begin, make sure you have the following installed on your system:

Required Software

  1. Python 3.11 or higher

Check your Python version: 

python --version
# or
python3 --version

If you need to install Python, visit python.org

  1. Git

Check if Git is installed: 

git --version

If you need to install Git, visit git-scm.com

  1. Text Editor or IDE

We recommend: - VS Code - PyCharm - Jupyter Notebook/Lab - Or any editor of your choice

  1. Ollama (for free, local LLM testing)

Install from ollama.ai

After installation, verify: 

ollama --version

Pull a recommended model: 

ollama pull qwen2.5:7b-instruct

Step 1: Clone the Repository

Clone the workshop repository to your local machine:

git clone https://github.com/ibm-client-engineering/output-drift-financial-llms
cd output-drift-financial-llms

For reproducibility, checkout the v0.1.0 release:

git checkout v0.1.0

Step 2: Set Up a Virtual Environment

Create and activate a Python virtual environment:

python3 -m venv venv
source venv/bin/activate

python -m venv venv
venv\Scripts\activate

You should see (venv) in your terminal prompt, indicating the virtual environment is active.

Step 3: Install Dependencies

Install all required Python packages:

pip install --upgrade pip
pip install -r requirements.txt

This will install: - ollama - Ollama API client for local models - pandas, numpy - Data analysis and numerical computing - matplotlib, scipy - Visualization and statistical analysis - rapidfuzz, python-Levenshtein - Drift detection metrics - sec-edgar-downloader, beautifulsoup4 - SEC filing data - faker - Synthetic financial data generation - python-dotenv - Environment variable management - pytest - Testing framework - And other dependencies (see requirements.txt for full list)

Step 4: Verify Installation

Test that the framework can import correctly:

python -c "from harness.task_definitions import format_rag_prompt, validate_sql_query; print('Installation successful')"

If you see "Installation successful", you're all set!

Step 5: Set Up API Keys (Optional)

To run experiments with cloud providers, you'll need API keys. Don't worry if you don't have all of these—you can start with Ollama (free and local) and add others later.

Create a .env File

In the repository root, create a .env file:

touch .env

Add Your API Keys

Edit the .env file and add the keys you have:

# Ollama (local, no key needed)
OLLAMA_BASE_URL=http://localhost:11434

# IBM watsonx.ai (if you have access)
WATSONX_API_KEY=your_watsonx_api_key_here
WATSONX_PROJECT_ID=your_project_id_here

# Anthropic (if you have access)
ANTHROPIC_API_KEY=your_anthropic_api_key_here

# Google Gemini (if you have access)
GEMINI_API_KEY=your_gemini_api_key_here

Keep Your Keys Secret

Never commit your .env file to Git! It's already in .gitignore to prevent accidental commits.

Where to Get API Keys

No API key needed! Just install and run locally.

# Install from https://ollama.ai/
ollama serve
  1. Visit watsonx.ai
  2. Sign up for a trial or use your IBM Cloud account
  3. Create a project and get your API key and project ID
  4. Setup Guide
  1. Visit console.anthropic.com
  2. Sign up and navigate to API keys
  3. Create a new API key
  4. Add billing information
  1. Visit aistudio.google.com
  2. Sign in with your Google account
  3. Create an API key
  4. Enable billing if needed for higher rate limits

Step 6: Test Your Setup

Test your providers to verify everything is configured correctly:

Test Ollama (Local)

If you installed Ollama:

# Check if Ollama is running
curl http://localhost:11434/api/tags

# Test with a simple query (requires qwen2.5:7b-instruct)
python -c "
from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')
response = client.chat.completions.create(
    model='qwen2.5:7b-instruct',
    messages=[{'role': 'user', 'content': 'Say hello!'}],
    temperature=0.0
)
print('✅ Ollama is working!')
print(f'Response: {response.choices[0].message.content}')
"

Test watsonx.ai (Optional)

If you configured watsonx.ai:

python -c "
import os
from dotenv import load_dotenv
load_dotenv()
print('✅ watsonx API key:', 'configured' if os.getenv('WATSONX_API_KEY') else 'NOT configured')
print('✅ watsonx Project ID:', 'configured' if os.getenv('WATSONX_PROJECT_ID') else 'NOT configured')
"

At Least One Provider

You need at least one provider configured (even just Ollama) to complete the workshop!

Troubleshooting

Python Version Issues

If you have multiple Python versions installed:

# Use python3.11 explicitly
python3.11 -m venv venv

Ollama Connection Issues

If Ollama isn't responding:

# Start Ollama server
ollama serve

# In another terminal, test:
curl http://localhost:11434/api/tags

Import Errors

If you see import errors, ensure you're in the virtual environment:

# Check if venv is activated
which python
# Should show: /path/to/your/repo/venv/bin/python

# If not, activate it:
source venv/bin/activate  # macOS/Linux
# or
venv\Scripts\activate  # Windows

Permission Errors on macOS/Linux

If you encounter permission errors:

# Use --user flag
pip install --user -r requirements.txt

Next Steps

Once your environment is set up and tested:

  1. Proceed to Lab 1: Understanding Output Drift
  2. Explore the examples/ directory for sample configurations
  3. Review the research paper in docs/resources/paper.md

Getting Help

If you encounter issues:

  1. Check the Troubleshooting Guide
  2. Ask workshop facilitators
  3. Open an issue on GitHub

Ready?

If all tests pass, you're ready to move on to Lab 1: Understanding Output Drift!