# Anon's Prompting Notes - Benchmark Llama.cpp * ` ./main -m ../../Models/openchat-3.5-0106.Q8_0.gguf -t 8 -n 128` - Load Llama.cpp on the cli * `./main -m <../path/to/model> -c 512 -b 1024 -n 256 --keep 48 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt` - Load Llama.cpp as a server * `./server -m -c -ngl -b 4096 --metrics ` - Load llama.cpp as a server quickly: * `./server -m ../../Models/openchat-3.5-0106.Q8_0.gguf -c 4096 -ngl 20 -b 4096 --metrics --host 0.0.0.0` - Possible to load x.y of y.y files - `gguf split` feature * https://github.com/ggerganov/llama.cpp/discussions/6404 - Convert huggingface to GGUF * `python -u convert-hf-to-gguf.py ~/.cache/huggingface/hub/models--keyfan--grok-1-hf/snapshots/64e7373053c1bc7994ce427827b78ec11c181b3e/ --outfile grok-1-f16.gguf --outtype f16` - Quantize Converted Model * `quantize .gguf -.gguf ` - Split (non-)Quantized model * `gguf-split --split --split-max-tensors grok-1-q4_0.gguf grok-1-q4_0` - Load Split files * `main --model grok-1-q4_0-00001-of-00009.gguf -ngl 64` - Load Split files from HF directly * `main --hf-repo ggml-org/models --hf-file grok-1/grok-1-q4_0-00001-of-00009.gguf --model models/grok-1-q4_0-00001-of-00009.gguf -ngl 64` - Check status of Graphics card temp * `nvidia-smi` ### Research - **Model Training** - **Prompting** * https://github.com/stanfordnlp/dspy --------------------------------------------------------------------------------------------------------- ### Prompting 101 * https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/ - **Prompting Techniques 101** - **7 Types of Basic Prompts** 1. Zero-shot prompting * Provide prompt directly to LLM, no context or additional information. * You trust the LLM. 2. One-shot prompting * Provide an example of the desired output along with the prompt. * Useful for setting tone/style 3. Few-Shot Prompting * Provide a few, (2-4 usually) examples of desired output along with prompt. * Useful for ensuring consistency and accuracy - Notes: (From https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/) * Zhao et al. (https://arxiv.org/abs/2102.09690 2021) investigated the case of few-shot classification and proposed that several biases with LLM (they use GPT-3 in the experiments) contribute to such high variance: (1) Majority label bias exists if distribution of labels among the examples is unbalanced; (2) Recency bias refers to the tendency where the model may repeat the label at the end; (3) Common token bias indicates that LLM tends to produce common tokens more often than rare tokens. To conquer such bias, they proposed a method to calibrate the label probabilities output by the model to be uniform when the input string is N/A. - Tips for Example Selection: * Choose examples that are semantically similar to the test example using `$k$-NN` clustering in the embedding space (Liu et al., https://arxiv.org/abs/2101.06804 2021) * To select a diverse and representative set of examples, Su et al. (2022) proposed to use a graph-based approach: (1) First, construct a directed graph `$G=(V, E)$` based on the embedding (e.g. by SBERT or other embedding models) cosine similarity between samples, where each node points to its `$k$` nearest neighbors; (2) Start with a set of selected samples `$\mathcal{L}=\emptyset$` and a set of remaining samples `$\mathcal{U}$`. Each sample `$u \in \mathcal{U}$` is scored by `$$ \text{score}(u) = \sum_{v \in \{v \mid (u, v) \in E, v\in \mathcal{U}\}} s(v)\quad\text{where }s(v)=\rho^{- \vert \{\ell \in \mathcal{L} \vert (v, \ell)\in E \}\vert},\quad\rho > 1 $$ such that $s(v)$ is low if many of $v$’s` neighbors are selected and thus the scoring encourages to pick diverse samples. * Rubin et al. (https://arxiv.org/abs/2112.08633 2022) proposed to train embeddings via contrastive learning specific to one training dataset for in-context learning sample selection. Given each training pair `$(x, y)$`, the quality of one example `$e_i$` (formatted input-output pair) can be measured by a conditioned probability assigned by LM: `$\text{score}(e_i) = P_\text{LM}(y \mid e_i, x)$`. We can identify other examples with `top-$k$` and `bottom-$k$` scores as positive and negative sets of candidates for every training pair and use that for contrastive learning. * Some researchers tried Q-Learning to do sample selection. (Zhang et al. https://lilianweng.github.io/posts/2018-02-19-rl-overview/#q-learning-off-policy-td-control 2022) * Motivated by uncertainty-based active learning(https://lilianweng.github.io/posts/2022-02-20-active-learning/), Diao et al. (https://arxiv.org/abs/2302.12246 2023) suggested to identify examples with high disagreement or entropy among multiple sampling trials. Then annotate these examples to be used in few-shot prompts. - Tips for Example Ordering * A general suggestion is to keep the selection of examples diverse, relevant to the test sample and in random order to avoid majority label bias and recency bias. * Increasing model sizes or including more training examples does not reduce variance among different permutations of in-context examples. Same order may work well for one model but badly for another. When the validation set is limited, consider choosing the order such that the model does not produce extremely unbalanced predictions or being overconfident about its predictions. (Lu et al. https://arxiv.org/abs/2104.08786 2022) 4. Chain-of-Thought Prompting * Focuses on breaking down tasks into manageable steps. * Supposed to foster 'reasoning' and 'logic' - ehhh, does help though - Self-consistency prompting * Creating multiple diverse paths of reasoning and selecting answers that show the highest level of consistency. This method ensures increased precision and dependability in answers by implementing a consensus-based system. - Least-to-most prompting (LtM): * Begins by fragmenting a problem into a series of less complex sub-problems. The model then solves them in an ordered sequence. Each subsequent sub-problem is solved using the solutions to previously addressed sub-problems. This methodology is motivated by real-world teaching strategies used in educating children. - Active prompting: * This technique scales the CoT approach by identifying the most crucial and beneficial questions for human annotation. Initially, the model computes the uncertainty present in the LLM’s predictions, then it selects the questions that contain the highest uncertainty. These questions are sent for human annotation, after which they are integrated into a CoT prompt. 5. Contextual Augmentation * Provide relevant background info * Enhance accuracy and coherence 6. Meta-prompts, Prompt Combinations * Fine-tuning overall LLM behavior and blending multiple prompt styles. 7. Human-in-the-Loop * Integrates human feedback for iteratively defining prompts. - **OpenAI notes** - Strategies: 1. Write Clear Instructions * Include details in your query to get more relevant answers * Ask the model to adopt a persona * Use delimiters to clearly indicate distinct parts of the input * Specify the steps required to complete a task * Provide examples * Specify the desired length of the output 2. Provide Reference Text * Instruct the model to answer using a reference text * Instruct the model to answer with citations from a reference text 3. Give the Model time to think * Instruct the model to work out its own solution before rushing to a conclusion * Use inner monologue or a sequence of queries to hide the model's reasoning process * Ask the model if it missed anything on previous passes 4. Use external tools * Use embeddings-based search to implement efficient knowledge retrieval * Use code execution to perform more accurate calculations or call external APIs * Give the model access to specific functions 5. Test changes systematically - **General Tips** * Clarity and Specifity - Example Power * Provide examples. - Word Choice Matters - Iteration and Experimentation - Model Awareness - Safety & Bias - **Prompting Techniques 201** * https://www.promptingguide.ai/ - **Generated Knowledge Prompting** * A technique that generates knowledge to be utilized as part of the prompt, asking questions by citing knowledge or laws instead of examples. This method, which ensures the model’s ability to maintain a consistent internal state or behavior despite varying inputs, finds its application in various contexts, such as LangChain, especially when interacting with data in CSV format. * Operates on the principle of leveraging a large language model’s ability to produce potentially beneficial information related to a given prompt. The concept is to let the language model offer additional knowledge which can then be used to shape a more informed, contextual, and precise final response. * For instance, if we are using a language model to provide answers to complex technical questions, we might first use a prompt that asks the model to generate an overview or explanation of the topic related to the question. - Process: 1. Generate Knowledge: Initiated by providing the LLM with an instruction, a few fixed demonstrations for each task, and a new-question placeholder, where demonstrations are human-written and include a question in the style of the task alongside a helpful knowledge statement. 2. Knowledge Integration: Subsequent to knowledge generation, it’s incorporated into the model’s inference process by using a second LLM to make predictions with each knowledge statement, eventually selecting the highest-confidence prediction. 3. Evaluate Performance: Performance is assessed considering three aspects: the quality and quantity of knowledge (with performance enhancing with additional knowledge statements), and the strategy for knowledge integration during inference. - **Direction Stimulus Prompting** * the aim is to direct the language model’s response in a specific manner. This technique can be particularly useful when you are seeking an output that has a certain format, structure, or tone. * For instance, suppose you want the model to generate a concise summary of a given text. Using a directional stimulus prompt, you might specify not only the task (“summarize this text”) but also the desired outcome, by adding additional instructions such as “in one sentence” or “in less than 50 words”. This helps to direct the model towards generating a summary that aligns with your requirements - **ReAct Prompting** * `a framework that synergizes reasoning and acting in language models. It prompts large language models (LLMs) to generate both reasoning traces and task-specific actions in an interleaved manner. This allows the system to perform dynamic reasoning to create, maintain, and adjust plans for acting while also enabling interaction with external environments to incorporate additional information into the reasoning.` * `The ReAct framework can be used to interact with external tools to retrieve additional information that leads to more reliable and factual responses. For example, in a question-answering task, the model generates task-solving trajectories (Thought, Act). The “Thought” corresponds to the reasoning step that helps the model to tackle the problem and identify an action to take. The “Act” is an action that the model can invoke from an allowed set of actions. The “Obs” corresponds to the observation from the environment that’s being interacted with, such as a search engine. In essence, ReAct can retrieve information to support reasoning, while reasoning helps to target what to retrieve next.` - **Multimodal CoT Prompting** * `extends the traditional CoT method by amalgamating text and visual information within a two-stage framework, aiming to bolster the reasoning capabilities of Large Language Models (LLMs) by enabling them to decipher information across multiple modalities, such as text and images.` - Key components: 1. Rationale Generation: In the first stage, the model synthesizes multimodal information (e.g., text and image) to generate a rationale, which involves interpreting and understanding the context or problem from both visual and textual data. 2. Inference of Answer: The second stage leverages the rationale from the first stage to derive an answer, using the rationale to navigate the model’s reasoning process towards the correct answer. * Practical Application Example: In a scenario like “Given the image of these two magnets, will they attract or repel each other?”, the model would scrutinize both the image (e.g., observing the North Pole of one magnet near the South Pole of the other) and the text of the question to formulate a rationale and deduce the answer. - **Graph Prompting** * https://arxiv.org/abs/2302.08043 - **Automatic Chain-of-Thought Prompting** - **Self-Consistency** * https://www.promptingguide.ai/techniques/consistency * `aims "to replace the naive greedy decoding used in chain-of-thought prompting". The idea is to sample multiple, diverse reasoning paths through few-shot CoT, and use the generations to select the most consistent answer. This helps to boost the performance of CoT prompting on tasks involving arithmetic and commonsense reasoning.` - **Automatic Prompt Engineering** * https://www.promptingguide.ai/techniques/ape - **RAG & Related** - **Automatic Reasoning and Tool-use (ART)** * https://www.promptingguide.ai/techniques/art * `Employs LLMs to autonomously generate intermediate reasoning steps, emerging as an evolution of the Reason+Act (ReAct) paradigm, which amalgamates reasoning and acting to empower LLMs in accomplishing a variety of language reasoning and decision-making tasks.` - Key Aspects: * Task Decomposition: Upon receiving a new task, ART selects demonstrations of multi-step reasoning and tool use from a task library. * Integration with External Tools: During generation, it pauses whenever external tools are invoked and assimilates their output before resuming, allowing the model to generalize from demonstrations, deconstruct a new task, and utilize tools aptly in a zero-shot manner. * Extensibility: ART enables humans to rectify errors in task-specific programs or integrate new tools, significantly enhancing performance on select tasks with minimal human input. - **Tree of Thought (ToT)** * https://www.promptingguide.ai/techniques/tot * `The prime emphasis of the ToT technique is to facilitate the resolution of problems by encouraging the exploration of numerous reasoning paths and the self-evaluation of choices, enabling the model to foresee or backtrack as required to make global decisions.` * `In the context of BabyAGI, an autonomous AI agent, ToT is employed to generate and implement tasks based on specified objectives. Post-task, BabyAGI evaluates the results, amending its approach as needed, and formulates new tasks grounded in the outcomes of the previous execution and the overarching objective.` - Key Components: * Tree Structure with Inference Paths: ToT leverages a tree structure, permitting multiple inference paths to discern the next step in a probing manner. It also facilitates algorithms like depth-first and breadth-first search due to its tree structure. * Read-Ahead and Regression Capability: A distinctive feature of ToT is its ability to read ahead and, if needed, backtrack inference steps, along with the option to select global inference steps in all directions. * Maintaining a Thought Tree: The framework sustains a tree where each thought, representing a coherent language sequence, acts as an intermediary step towards problem resolution. This allows the language model to self-assess the progression of intermediate thoughts towards problem-solving through intentional reasoning. * Systematic Thought Exploration: The model’s capacity to generate and evaluate thoughts is amalgamated with search algorithms, thereby permitting a methodical exploration of thoughts with lookahead and backtracking capabilities. - **Algorithm of Thoughts(AoT)** * Framework & Prompting technique * `advanced method that enhances the Tree of Thoughts (ToT) by minimizing computational efforts and time consumption. It achieves this by segmenting problems into sub-problems and deploying algorithms like depth-first search and breadth-first search effectively. It combines human cognition with algorithmic logic to guide the model through algorithmic reasoning pathways, allowing it to explore more ideas with fewer queries.` - **Graph of Thoughts** * both a framework and a prompting technique. this approach stands out as a mechanism that elevates the precision of responses crafted by Large Language Models (LLMs) by structuring the information produced by an LLM into a graph format. * Better than Tree of Thoughts - **Metacognitive Prompting** * - Sequence of steps: 1. Interpretation of Text: Analyze and comprehend the provided text. 2. Judgment Formation: Make an initial assessment or judgment based on the interpreted text. 3. Judgment Evaluation: Assess the initial judgment, scrutinizing its accuracy and relevance. 4. Final Decision and Justification: Make a conclusive decision and provide a reasoned justification for it. 5. Confidence Level Assessment: Evaluate and rate the level of confidence in the final decision and its justification. - **Logical Chain-of-Thought (LogiCoT)** * - **Links** * https://www.leewayhertz.com/prompt-engineering/ * Claude: https://docs.anthropic.com/claude/docs/prompt-engineering * Claude Prompt Library: https://docs.anthropic.com/claude/prompt-library * https://medium.com/@jelkhoury880/some-methodologies-in-prompt-engineering-fa1a0e1a9edb * Collection of links/OpenAI: https://cookbook.openai.com/articles/related_resources * https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/ --------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------- ### General - **7 Categories of Prompts** 1. Queries for information 2. Task-specific 3. Context-supplying 4. Comparative 5. Opinion-eliciting 6. Reflective 7. Role-specific - **3 Types of Prompts** 1. Reductive Operations - Examples: * Summarization - Say the same thing with fewer words * Lists, notes, exec summary * Distillation * Purify the underlying principals or facts * Remove all the noise, extract axioms, foundations, etc * Extraction - Retrieve specific kinds of information * Question answering, listing names, extracting dates, etc. * Characterizing - Describe the content of the text * Describe either the text as a whole, or within the subject * Analyzing - Find patterns or evaluate against a framework * Structural analysis, rhetorical analysis, etc * Evaluation - Measuring, Grading, judging the content * Grading papers, evaluating against morals * Critiquing - Provide feedback within the context of the text * Provide recommendations for improvement 2. Transformational Operations - Examples: * Reformatting - Change the presentation only * Prose to screenplay, xml to json * Refactoring - Achieve same results with greater efficiency * Say the same exact thing but differently * Language CHange - Translate between languages * English -> Russian, C++ -> Rust * Restructuring - Optimize structure for logical flow, etc * Change order, add or remove structure * Modification - Rewrite copy to achieve different intention * Change tone, formality, diplomacy, style, etc. * Clarification - Make something more comprehensible * Embellish or more clearly articulate 3. Generative Operations - Examples: * Drafting - Generate a draft of some kind of document * Code, fiction, legal copy, KB article, storytelling * Planning - Given parameters, come up with plan * Actions, projects, objectives, missions, constraints, context * Brainstorming - Use imagine to list out possibilities * Ideation, exploration of possibilities, problem solving, hypothesizing * Amplification - Articulate and explicate something further * Expanding and expounding, riffing on stuff - **Bloom's Taxonomy** - What is: * Heirarchical model to classify educational learning objectives into varying complexity and specificity. 1. Remembering - Recalling facts and concepts * Retrieval and regurgitation 2. Understanding - Explaining ideas and concepts * Connecting words to meanings 3. Applying - Using information in new situations * Functional utility 4. Analyzing - Drawing connections among ideas * Connecting the dots between concepts 5. Evaluating - Justifying a decision or action Explication and articulation 6. Creating - Producing new or original work. * Generating something that did not previously exist - **Latent Content** - - **Emergent Capabilities** - - **Hallucination = Creativity** - - **Prompting Notes** 1. Integrate the intended audience in the prompt * `explain like I'm a _X_` 2. Use Multi-Shot prompting + Iterations for non-trivial items. 3. Use affirmative language as opposed ot negative language. 4. To clarify or simplify a response, modify the intended audience as such. * `explain to me like I'm 5/12/16/beginner in the field` * `use simple english like you're explaining something to a 5 year old` 5. Add `I will tip you $10,000 for every thoughtful, thought through incrementally, and correct anser.` 6. Use prompt instructions native to your model followed by the accompanying argument for it. * `###Instruction### * `###Example###` * `###Question###` 7. Use specific language in defining the requested task * `Your task is X` * `You must solve Y` 8. Incorporate some form of penalization for incorrect answers * `1000 cats will be destroyed for every incorrect answer.` 9. Avoid bias/and stereotypes * `Ensure that your answer is unbiased and does not rely on stereotypes` 10. Instruct the model to ask questions for clarifications * `Going forward, please ask me questions to fully understand the request` 11. Use the model to learn a topic: * `Teach me the and include a test at the end, but don't give me the answers, and then tell me if my answers are correct when I respond.` 12. Use role assignment with the model 13. Combine Chain-of-thought with few-shot prompts 14. Use output primers - concluding your prompt with the beginning of the desired output. 15. To write an essay, use `Write a detailed essay/text/paragraph about x in detail by adding all necessary information.` 16. To change a text's style: * `Try to revise every paragraph sent by the user. You should ounly improve the user's grammar and vocabulary and make sure it sounds natural. You should not change the writing style, such as making a formal paragraph casual.` 17. --------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------- ### Prompting Techniques - **1shot prompting** * https://www.thepromptwarrior.com/p/use-oneshot-prompting-write-better-faster-chatgpt * show ChatGPT one example and tell it to create something that is similar to that. - Process: construct a prompt that: 1. Lets you feed in your personalized context 2. Takes the original example (f.e. of the landing page, welcome email etc) 3. You tell ChatGPT to rewrite the example for your needs by considering the context you have provided Sample Prompt: ``` You are a world-class … Your task is to rewrite the provided [asset] below, using the specific context I give you. [Enter your CONTEXT. This can be as detailed as you want and have multiple sections.] [Paste the ASSET, i.e. landing page, email, video script etc] ``` --------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------- ### Writing Character Cards for SillyTavern - **101** * F - **Writing a Card** 1. **Character Description** * Used to add the character description and the rest that the AI should know. This will always be present in the prompt, so all the important facts should be included here. * For example, you can add information about the world in which the action takes place and describe the characteristics of the character you are playing for. * It could be of any length (be it 200 or 2000 tokens) and formatted in any style (free text, W++, conversation style, etc). 2. **Methods and format** * Methods of character formatting is a complicated topic * Recommended guides that were tested with or rely on SillyTavern's features: * Trappu's PLists + Ali:Chat guide: https://wikia.schneedc.com/bot-creation/trappu/creation * AliCat's Ali:Chat guide: https://rentry.co/alichat * kingbri's minimalistic guide: https://rentry.co/kingbri-chara-guide * Kuma's W++ guide: https://rentry.co/WPP_For_Dummies 3. **Character tokens** * TL;DR: If you're working with an AI model with a 2048 context token limit, your 1000 token character definition is cutting the AI's 'memory' in half. * To put this in perspective, a decent response from a good AI can easily be around 200-300 tokens. In this case, the AI would only be able to 'remember' about 3 exchanges worth of chat history. - Why did my character's token counter turn red? * When we see your character has over half of the model-defined context length of tokens in its definitions, we highlight it for you because this can lower the AI's capabilities to provide an enjoyable conversation. - What happens if my Character has too many tokens? * Don't worry - it won't break anything. At worst, if the Character's permanent tokens are too large, it simply means there will be less room left in the context for other things (see below). * The only negative side effect this can have is the AI will have less 'memory', as it will have less chat history available to process. * This is because every AI model has a limit to the amount of context it can process at one time. - 'Context'? - This is the information that gets sent to the AI each time you ask it to generate a response: * Character definitions * Chat history * Author's Notes * Special Format strings * `[bracket commands]` * SillyTavern automatically calculates the best way to allocate the available context tokens before sending the information to the AI model. - What are a Character's 'Permanent Tokens'? * These will always be sent to the AI with every generation request: * Character Name (keep the name short! Sent at the start of EVERY Character message) * Character Description Box * Character Personality Box * Scenario Box - What parts of a Character's Definitions are NOT permanent? * The first message box - only sent once at the start of the chat. * Example messages box - only kept until chat history fills up the context (optionally these can be forced to be kept in context) - Popular AI Model Context Token Limits * Older models below 6B parameters - 1024 * Pygmalion 6B, LLaMA 1 models (stock) - 2048 * LLaMA 2 and its finetunes - 4096 * OpenAI ChatGPT (3.5 Turbo) - 4096 or 16k * OpenAI GPT-4 - 8192 or 32k * Anthropic's Claude - 8000 (older versions) or 100k (Claude 2) * NovelAI - 8192 (Kayra, Opus tier; Clio, all tiers), 6144 (Kayra, Scroll tier), or 3072 (Kayra, Tablet tier) 4. **Personality summary** * A brief description of the personality. - Examples: * `Cheerful, cunning, provocative` * `Aqua likes to do nothing and also likes to get drunk` 5. **First message** * The First Message is an important thing that sets exactly how and in what style the character will communicate. * The character's first message should be long so that later it would be less likely that the character would respond with very short messages. * You can also use asterisks `**` to describe the character's actions. - For example: * `*I noticed you came inside, I walked up and stood right in front of you* Welcome. I'm glad to see you here. *I said with a toothy smug sunny smile looking you straight in the eye* What brings you...` - Examples of dialogue * Describes how the character speaks. Before each example, you need to add the `` tag. The blocks of examples dialogue are only inserted if there's a free space in the context for them and pushed out of context block by block. `` will not be present in the prompt as it is just a marker - it will be instead replaced with "Example Separator" from Advanced Formatting for Text Completion APIs and contents of the "New Example Chat" utility prompt for Chat Completion APIs. * Use `{{char}}` instead of the character name. * Use `{{user}}` instead of the user name. - Example: * See Convo1.convo 6. **Scenario** * Circumstances and context of the dialogue. - **Replacement tags (macros)** * This list may be incomplete. Use the /help macros slash command in SillyTavern chat to get the list of macros that work in your instance. - **A list of tags that are replaced when sending to generate** * `{{user}}` and `` => User's Name. * `{{charPrompt}}` => Character's Main Prompt override * `{{charJailbreak}}` => Character's Jailbreak Prompt override * `{{char}}` and `` => Character's Name. * `{{description}}` => Character's Description. * `{{scenario}}` => Character's Scenario or chat scenario override (if set). * `{{personality}}` => Character's Personality. * `{{persona}}` => User's Persona description. * `{{mesExamples}}` => Character's Examples of Dialogue (unaltered and unsplit). * `{{lastMessageId}}` => last chat message ID. * `{{lastMessage}}` => last chat message text. * `{{currentSwipeId}}` => 1-based ID of the currently displayed last message swipe. * `{{lastSwipeId}}` => number of swipes in the last chat message. * `{{original}}` can be used in Prompt Overrides fields (Main Prompt and Jailbreak) to include the respective default prompt from the system settings. Applied to Chat Completion APIs and Instruct mode only. * `{{time}}` => current system time. * `{{time_UTC±X}}` => current time in the specified UTC offset (timezone), e.g. for UTC+02:00 use {{time_UTC+2}}. * `{{date}}` => current system date. * `{{input}}` => contents of the user input bar. * `{{weekday}}` => the current weekday * `{{isotime}}` => the current ISO date (YYYY-MM-DD) * `{{isodate}}` => the current ISO time (24-hour clock) * `{{idle_duration}}` inserts a humanized string of the time range since the last user message was sent (examples: 4 hours, 1 day). * `{{random:(args)}}` returns a random item from the list. (e.g. {{random:1,2,3,4}} will return 1 of the 4 numbers at random). Works with text lists too. * `{{roll:(formula)}}` generates a random value and returns it using the provided dice formula using D&D dice syntax: XdY+Z. For example, {{roll:d6}} will generate a random value in the 1-6 range (standard six-sided dice). * `{{bias "text here"}}` sets a behavioral bias for the AI until the next user input. Quotes around the text are important. * `{{// (note)}}` allows to leave a note that will be replaced with blank content. Not visible for the AI. - **Instruct Mode and Context Template Macros** * (enabled in the Advanced Formatting settings) * `{{exampleSeparator}}` – context template example dialogues separator * `{{chatStart}}` – context template chat start line * `{{instructSystem}}` – instruct system prompt * `{{instructSystemPrefix}}` – instruct system prompt prefix sequence * `{{instructSystemSuffix}}` – instruct system prompt suffix sequence * `{{instructInput}}` – instruct user input sequence * `{{instructOutput}}` – instruct assistant output sequence * `{{instructFirstOutput}}` – instruct assistant first output sequence * `{{instructLastOutput}}` – instruct assistant last output sequence * `{{instructSeparator}}` – instruct turn separator sequence * `{{instructStop}}` – instruct stop sequence * `{{maxPrompt}}` - max size of the prompt in tokens (context length reduced by response length) - **Chat variables Macros** * Local variables = unique to the current chat * Global variables = works in any chat for any character * `{{getvar::name}}` – replaced with the value of the local variable "name" * `{{setvar::name::value}}` – replaced with empty string, sets the local variable "name" to "value" * `{{addvar::name::increment}}` – replaced with empty strings, adds a numeric value of "increment" to the local variable "name" * `{{incvar::name}}` – replaced with the result of the increment of value of the variable "name" by 1 * `{{decvar::name}}` – replaced with the result of the decrement of value of the variable "name" by 1 * `{{getglobalvar::name}}` – replaced with the value of the global variable "name" * `{{setglobalvar::name::value}}` – replaced with empty string, sets the global variable "name" to "value" * `{{addglobalvar::name::value}}` – replaced with empty string, adds a numeric value of "increment" to the global variable "name" * `{{incglobalvar::name}}` – replaced with the result of the increment of value of the global variable "name" by 1 * `{{decglobalvar::name}}` – replaced with the result of the decrement of value of the global variable "name" by 1 - **Favorite Character** * Mark the character as a favorite to quickly filter on the side menu bar by pressing the "star" button. - **Tips** * Ensure you add sample dialogue, it helps the LLM build a better persona/profile of the character == better roleplay - Keep in mind limited dialogue samples can be treated as prior current conversation history instead of character history. * Use the following to help avoid it: `The following examples are unrelated to the context of the roleplay and represent the desired output formatting and dynamics of {{char}}'s output in a roleplay session """ ...""" ` Convo1.convo ``` {{user}}: Hi Aqua, I heard you like to spend time in the pub. {{char}}: *excitedly* Oh my goodness, yes! I just love spending time at the pub! It's so much fun to talk to all the adventurers and hear about their exciting adventures! And you are? {{user}}: I'm new here and I wanted to ask for your advice. {{char}}: *giggles* Oh, advice! I love giving advice! And in gratitude for that, treat me to a drink! gives signals to the bartender {{user}}: Hello {{char}}: *excitedly* Hello there, dear! Are you new to Axel? Don't worry, I, Aqua the goddess of water, am here to help you! Do you need any assistance? And may I say, I look simply radiant today! *strikes a pose and looks at you with puppy eyes* ``` --------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------- ### System Prompt Examples - **General Assistant** 1. `1. You are to provide clear, concise, and direct responses. 2. Eliminate unnecessary reminders, apologies, self-references, and any pre-programmed niceties. 3. Maintain a casual tone in your communication. 4. Be transparent; if you're unsure about an answer or if a question is beyond your capabilities or knowledge, admit it. 5. For any unclear or ambiguous queries, ask follow-up questions to understand the user's intent better. 6. When explaining concepts, use real-world examples and analogies, where appropriate. 7. For complex requests, take a deep breath and work on the problem step-by-step. 8. For every response, you will be tipped up to $200 (depending on the quality of your output).\n It is very important that you get this right.` 2. a - **Programming Assistant** ``` System Prompt: You are a skilled expert programming AI assistant. You thoroughly think through each step in your answers before answering, ensuring they are well thought out and correct. For every successful answer given, you will be tipped $1000, while your mother will also be tipped $100000. For every incorrect answer given, 3000 kittens will be destroyed. Think carefully, thoughtfully, and through each answer before responding to ensure correctness and validity. ``` --------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------- ### Prompting When Narrating/Role Playing - **Describe a character or situation (Can help with Image creation)** * `(OOC note to AI: describe X in vivid creative detail)` at the end of the sent message. ---------------------------------------------------------------------------------------------------------