Skip to main content

Log 5 🛫

· 2 min read

Objectives​

  • Deploy watsonx.ai on self-managed AWS infrastructure.

Accomplishments​

AWS

  • Populated parameter overrides JSON.
  • Created RH Trial account and uploaded pull secret to S3 bucket.
  • Updated CloudFormation STS template with permissions to create and assume Role with respective JSON versions.

RAG

  • Creation of cronjob to capture logs from Python app.
  • Enabled metadata insertion into chunks in vector store -> (hopefully) increases retrieval accuracy
  • Return context to user (shows sources used to generate responses)
  • Added mixtral model support
  • Enable functionality for user to give custom rag parameters
  • Migrated vector DB from FAISS to chromaDB to enable the metadata functionality
  • Script written to easily test rag implementation and save results in csv
  • Implemented cache logic to make sure it considers combination of parameters as well before chosing to send a cached response

In Progress​

  • End-to-end deployment of OCP, CP4D, and watsonx.ai (with GPU node)
  • Tagging cp-deployer.sh generated resources.
  • Updating solution docs with better asset linking.

Next Steps​

  • Continue over the shoulder working sessions
    • Kick off CloudFormation template install with updated STS templates.
  • Compilation of required endpoints
  • Deploy latest RAG version on AWS
  • Build out actions & flow in Watsonx Assistant after properly defining personas & objectives.

Tracking (Issues)​

  • Require sign-off on final CloudFormation template.
  • CoreOS AMI pending approval.