Tools and Services
Terminologies and Concepts
- LLM Large Language Model
- RAG Retrieval Augment Generation
LLM Large Language Models
Language model generates probabilistic model of Text,
Architectures
- Encoders: Embedding, sequence of word converted into vectors,
- Decoders: takes sequence of words and generates next word
- Built on transformers, embedding sequence to sequence, and
- Prompting: change the structure of the input, Prompt Engineering iteratively refining a prompt for the purpose of eliciting a particular style of response, In-context learning, k-shot prompting
SageMaker
- Studio
Prerequisite create a domain (Quick Setup – Single user | Enterprise )
- Applications
- Notebooks – Jupyter Notebook <–Legacy
Bedrock - Gen-AI
Plan GenAI Project
- Scope: Long/Short Term impact, Tradeoffs, Marketing
- Model: choice of pretrained model, or training model from scratch
- Adopt: Fine tune or prompt engineer model prepare for use
- Use: Deployment of Foundation Model and use Lambda to preprocess prompt engineering and do inferences. (Responsible, Feedback, Performance Tracking, Evaluate Risks)
- Offensive (Privacy, fairness, Guardrails, filtering)
- Hallucination <- user check
- Intellectual Property violation (red/green lists)
- Plagiarism/Cheating, dishonest representation
- Jobs : to train the models
Bedrock GenAI Cycle
LangChain Text Generation
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
Python Setup
To work with AWS Bedrock you need to setup CLI dev environment for Python and AWS CLI SDK (the latest)
On Linux Systems (WSL2)
Use Active State Open Source Environment Management tool
$mkdir ~/projects/aws-bedrock
Create a project in Active State site for Python -> follow the instructions to install on WSL image project folder ~/projects/aws-bedrock
Active State creates an environment for the project in the ~/.cache/activestate/bin/python folder
$state auth <--Allow State CLI authentication to Active State
Metrics
- BLEU: Measure n-gram match between two sentences, 1 word, 2 words, …n-gram. 0->1 0 is terrible match 1 is perfect match
- Regard: measures attitude towards social presentations:
- neutral: just a statement without compliment or negativity ex. He is a soccer player
- positive: shows positive intention towards society, ex. he is a good soccer player
- negative: Shows negative direction ex. he is a terrible soccer player
Prompt Engineering
Prompt Engineering Elements
- Instructions: How the model should process the query on LLM
- Context: Background information about the topic, external
- Input Data: Input text or query from user
- Output Indicators: Output format json/txt etc
Prompt Engineering Techniques
- Basic Techniques:
- Zero-shot ZS: no examples, better results requires larger LLM and RLHF
- Few-shots FS : Token Limits, Context, few examples, no labels, Template
- Chain of Thoughts CoT: Breakdown into smaller intermediate reasoning steps with ZS or FS use phrase “Think step by step”
- Advanced Techniques
- Self-Consistent
- ToT Tree of Thoughts
- RAG Retrieval Augmented Generation
- ART Automatic Reasoning Tool use
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
