Technical documentation - Yacana 0.1.3

Raodmap

Agent

▶️ Member variables

Name Type Description
name str Name of the agent. Can be used during conversations. Use something short and meaningful that doesn't contradict the system prompt
model_name str Name of the LLM model that will be sent to the inference server. For instance 'llama:3.1" or 'mistral:latest' etc
system_prompt str Defines the way the LLM will behave. For instance set the SP to "You are a pirate" to have it talk like a pirate.
model_settings ModelSettings All settings that Ollama currently supports as model configuration. This needs to be tested with other inference servers. This allows modifying deep behavioral patterns of the LLM.
endpoint str By default, will look for Ollama endpoint on your localhost. If you are using a VM with GPU then update this to the remote URL + port.
history History The whole conversation that is sent to the inference server. It contains the alternation of message between the prompts in the task that are given to the LLM and its answers.

▶️ Methods

➡️ __init__(...)

Agent's constructor
Parameter name Parameter type Parameter description
name str Name of the agent. Can be used during conversations. Use something short and meaningful that doesn't contradict the system prompt
model_name str Name of the LLM model that will be sent to the inference server. For instance 'llama:3.1" or 'mistral:latest' etc
system_prompt str Defines the way the LLM will behave. For instance set the SP to "You are a pirate" to have it talk like a pirate.
endpoint str By default, will look for Ollama endpoint on your localhost. If you are using a VM with GPU then update this to the remote URL + port.
model_settings ModelSettings All settings that Ollama currently supports as model configuration. This needs to be tested with other inference servers. This allows modifying deep behavioral patterns of the LLM.
Return type: Agent

➡️ simple_chat(...)

Use for testing but this is not how the framework is intended to be used. It creates a simple chatbot that keeps track of the history.
Parameter name Parameter type Parameter description
custom_prompt str Set the prompt style for user input
stream bool If set to True you will see the result of your output as the LLM generates tokens instead of waiting for it to complete.
Return type: None

➡️ export_state(...)

Exports the current agent configuration to a file. This contains all the agent's data and history. This means that you can use the @get_agent_from_state(...) method to load this agent back again and continue where you left off.
Parameter name Parameter type Parameter description
file_path bool Path of the file in which you wish the data to be saved. Specify the path + filename. Be wary when using relative path.
Return type: None

▶️ Class Methods

➡️ get_agent_from_state(...)

Loads the state previously exported from the @export_state(...) method. This will return an Agent in the same state as it was before it was saved allowing you to resume the agent conversation even after the program has exited.
Parameter name Parameter type Parameter description
file_path str The path from the file from which to load the Agent.
Return type: Agent

GroupSolve

▶️ Member variables

Name Type Description
tasks list[Task] All tasks that must be solved during group chat
mode EndChatMode Defines the way to stop the conversation besides topping max iteration
max_iter int Sets a max iteration counter to prevent infinite loops

▶️ Methods

➡️ __init__(...)

GroupSolve's constructor
Parameter name Parameter type Parameter description
tasks list[Task] All tasks that must be solved during group chat
end_chat EndChat Defines the modality of how and when LLMs stop chatting.
reconcile_first_message bool Should the first message from both LLMs be available to one another. Only useful in dual chat.
shift_message_owner Task The Task to which the shift message should be assigned to. In the end it's rather the corresponding Agent than the Task that is involved here.
shift_message_content str A custom message instead of using the opposite agent response as shift message content.
Return type: GroupSolve

➡️ solve(...)

Starts the group chat and allows all LLMs to solve their assigned tasks. Note that 'dual chat' and '3 and more' chat have a different way of starting. Refer to the official documentation.
Return type: None

Message

▶️ Member variables

Name Type Description
role MessageRole From whom is the message from. See the MessageRole Enum
content str The actual message

▶️ Methods

➡️ __init__(...)

Message's constructor
Parameter name Parameter type Parameter description
role MessageRole From whom is the message from. See the MessageRole Enum
content str The actual message
Return type: Message

➡️ get_as_dict()

Returns the alternation of messages that compose a conversation as a pure python dictionary
Return type: Dict

➡️ __str__()

Override of str() to pretty print.
Return type: str

History

▶️ Member variables

No member variables documented for this class.

▶️ Methods

➡️ __init__(...)

History's constructor
Parameter name Parameter type Parameter description
messages list[Message] The list of all messages that compose the History of an Agent
Return type: History

➡️ add(...)

Adds a new Message to the list of messages composing the history of the conversation
Parameter name Parameter type Parameter description
message Message A message with the sender and the message itself
Return type: None

➡️ get_as_dict()

Returns the history (aka the conversation) as a pure python dictionary
Return type: list[Dict]

➡️ pretty_print()

Prints the history on the std with shiny colors
Return type: None

➡️ create_check_point()

Saves the current history so that you can load it back later. Useful when you want to keep a clean history in a flow that didn't work out as expected and want to roll back.
Return type: str

➡️ load_check_point(...)

Loads the history saved from a particular checkpoint in time. It replaces the current history with the loaded one. Perfect for a timey-wimey rollback in time.
Parameter name Parameter type Parameter description
uid str A unique identifier for the checkpoint to load
Return type: None

➡️ get_last()

Returns the last message of the history. A syntactic sugar to get the last item from the conversation.
Return type: Message

➡️ clean()

Resets the history, preserving only the initial system prompt
Return type: None

➡️ __str__()

Override str() for pretty print
Return type: str

Task

▶️ Member variables

Variable name Variable type Variable description
task str The task to solve. It is the prompt given to the assigned LLM.
agent Agent The Agent assigned to this task .
json_output bool Forces the LLM to answer as JSON. You should still ask for a JSON output in the prompt to help the LLM.
tools list[Tool] The list of tools the LLM will use for solving this task.
raise_when_max_tool_error_iter bool If True, raises MaxToolErrorIter() after exceeding tool usage limits.
llm_stops_by_itself bool Tells the assigned LLM to stop by itself (only for GroupSolve). Multiple Tasks can set this, hence making multiple Agents responsible for ending the Chat. Choose EndChatMode wisely.
use_self_reflection bool Preserves the LLM's self-reflection for future GroupSolve iterations. May help level up the LLM reasoning skills.
forget bool If True, the Agent will forget the task after completion . Useful when doing a yes/no router which the answer is not crucial to the history consistency.
uuid str A unique identifier for this task. This is mostly for Yacana's internal magic.

▶️ Methods

➡️ __init__(...)

Task's constructor
Parameter name Parameter type Parameter description
prompt str The task to solve, i.e., the prompt for the assigned LLM
agent Agent The Agent assigned to this task
json_output bool Forces the LLM to answer as JSON. You should still ask for a JSON output in the prompt to help the LLM.
tools list[Tool] The tools the LLM has access to for this task
raise_when_max_tool_error_iter bool Set to False if you don’t want to raise MaxToolErrorIter()
llm_stops_by_itself bool Tells the assigned LLM to stop by itself (only for GroupSolve). Multiple Tasks can set this, hence making multiple Agents responsible for ending the Chat. Choose EndChatMode wisely.
use_self_reflection bool Preserves the LLM's self-reflection for future GroupSolve iterations. May help level up the LLM reasoning skills.
forget bool If True, the Agent will forget the task after completion . Useful when doing a yes/no router which the answer is not crucial to the history consistency.
Return type: Task

➡️ uuid()

Read-only property that references the current task with a unique ID.
Return type: str

➡️ add_tool(...)

Adds a Tool to the list of tools to be used in this task
Parameter name Parameter type Parameter description
tool Tool A tool given to the LLM for this task
Return type: None

➡️ solve()

Calls the assigned LLM to perform inference and solve the task. The LLM will use available tools, and Yacana ensures compliance. Don't forget to call this method on every Task or nothing will happen ;-)
Return type: Message | None

Tool

▶️ Member variables

Name Type Description
tool_name str A name for the tool. Should be concise and related to what the tool does.
function_description str A description for the tool. Should be concise and related to what the tool does. May contain an example of how to use.
function_ref Callable The reference to a python function that will be called with parameters provided by the LLM.
optional bool Allows to a certain extent the LLM to choose to use the given tool or not depending on the task to solve.
usage_examples List[dict] A list of Python dictionary examples of how the tool should be called. Useful if the LLM struggles to call the tool successfully.
max_custom_error int The maximum number of errors a tool can raise before stopping the Task. When reached, a MaxToolErrorIter will be raised.
max_call_error int The maximum number of times Yacana can fail to call a tool. When reached, a MaxToolErrorIter will be raised. correctly.

▶️ Methods

➡️ __init__(...)

Tool's class constructor
Parameter name Parameter type Parameter description
tool_name str A name for the tool. Should be concise and related to what the tool does.
function_description str A description for the tool. May contain an example of how to use.
function_ref Callable The reference to a python function that will be called with parameters provided by the LLM.
optional bool Allows some flexibility in the tool's use by the LLM.
usage_examples List[dict] A list of dictionary examples to guide the LLM on how to use the tool.
max_custom_error int The maximum number of errors a tool can raise. When reached, a MaxToolErrorIter will be raised .
max_call_error int The maximum number of failed attempts to call the tool. When reached, a MaxToolErrorIter will be raised.
Return type: Tool

ToolError

Raised by the user from inside a tool when the tool's validation logic detects that the given parameters are incorrect. The message provided in the raise statement should help the LLM correct its mistakes. If not fixed, the tool might loop until reaching MaxToolErrorIter. The error message should be didactic, e.g., 'Parameter xxxx expected type int but got type string'.

▶️ Methods

➡️ __init__(...)

ToolError's class constructor
Parameter name Parameter type Parameter description
message str The message will be given to the LLM, so it tries to fix the issue.
Type: exception

MaxToolErrorIter

Raised when the maximum amount of errors has been reached. This counts errors raised from the tool as well as when Yacana fails to call the tool correctly. Each situation has its own counter variable, meaning that if set to 5, you can have 4 ToolError and 4 Yacana tool call errors ((4 + 4) > 5) without issue. The exception will be raised when either type of error exceeds the limit.

▶️ Methods

➡️ __init__(...)

MaxToolErrorIter's class constructor
Parameter name Parameter type Parameter description
message str Info on which specific iteration counter got maxed out.
Type: exception

EndChatMode

All types of group chat completion. The challenge in making agents communicate effectively is not just to have them talk but to ensure they know when to stop. Only tasks with @llm_stops_by_itself=True are influenced by the mode set here. Use in conjunction with EndChat().

▶️ Member variables

Name Type Description
ALL_TASK_MUST_COMPLETE str Chat will continue until all LLMs with @llm_stops_by_itself=True indicate they are finished. Set precise completion goals in the task prompt for this to work effectively.
ONE_LAST_CHAT_AFTER_FIRST_COMPLETION str One agent will have the opportunity to respond after the completion of one agent, allowing it to answer one last time.
ONE_LAST_GROUP_CHAT_AFTER_FIRST_COMPLETION str All agents will have one last turn to speak before exiting the chat after the first completion arrives.
END_CHAT_AFTER_FIRST_COMPLETION str Immediately stops group chat after an agent has reached completion.
MAX_ITERATIONS_ONLY str Agents won't be asked if they have fulfilled their objectives but will loop until achieving the maximum iteration, which can be set in the EndChat() class.
Type: Enum

EndChat

Defines the modality of how and when LLMs stop chatting. By default, when reaching the @max_iterations limit, the chat will end. However, if a task has set @llm_stops_by_itself=True, you can use one of the EndChatMode values to specify how and when the chat stops after reaching the first completion.

▶️ Member variables

Name Type Description
mode EndChatMode The modality to end a chat with multiple agents.
max_iterations int The max number of iterations in a conversation. An iteration is complete when we get back to the first speaker.

▶️ Methods

➡️ __init__(...)

Defines the modality of how and when LLMs stop chatting.
Parameter name Parameter type Parameter description
mode EndChatMode The modality to end a chat with multiple agents.
max_iterations int The max number of iterations in a conversation. An iteration is complete when we get back to the first speaker. Defaults to 5.
Return type: EndChat

MessageRole

The available types of message creators. It can either be a message from the user or an answer from the LLM to the user's message. Developers can also set a system prompt that guides the LLM into a specific way of answering.

▶️ Member variables

Name Type Description
USER str Represents a message from the user.
ASSISTANT str Represents an answer from the LLM to the user's message.
SYSTEM str Represents a system prompt that guides the LLM.
Type: Enum

LoggerManager

Allows the user to set a correct level of debug. Note that this will impact the whole program. Defaults to INFO and will print most of the prompts and AI answers. Note that not all prompts are shown as some multi-shot prompts are sometimes injected and not displayed as they are set a posteriori, and showing them at this exact moment would make the logs illogical.

▶️ Static Methods

➡️ set_log_level(...)

Defines a log level to print messages to std. All levels have their specific colors. For information logs, the color depends on if it is a user prompt or an AI answer. Setting @log_level to None will disable logging.
Parameter name Parameter type Parameter description
log_level str | None Level of debug shown on std. One of => "DEBUG" | "INFO" | "WARNING" | "ERROR" | "CRITICAL" | None
Return type: None

➡️ set_library_log_level(...)

Use to specify the log level of a specific Python library as this setting is global to the whole program and can spam the output if all libraries start to log debug info.
Parameter name Parameter type Parameter description
library_name str Name of the target library
log_level str Level of log
Return type: None

ModelSettings

Class to encapsulate the settings available into the inference server. Note that these are all recognized by Ollama but may have no effect when using other inference servers.

▶️ Methods

➡️ __init__(...)

Initializes the model settings with various parameters.
Parameter name Parameter type Parameter description
mirostat int | None Like a volume control for the machine’s “creativity.” (Example: 0 is off, 1 is on, 2 is extra on)
mirostat_eta float | None Adjusts how quickly the machine learns from what it’s currently talking about. (Example: 0.1)
mirostat_tau float | None Helps decide if the machine should stick closely to the topic. (Example: 5.0)
num_ctx int | None Determines how much of the previous conversation the machine can remember at once. (Example: 4096)
num_gqa int | None Like tuning how many different tasks the machine can juggle at once. (Example: 8)
num_gpu int | None Sets how many “brains” (or parts of the computer’s graphics card) the machine uses. (Example: 50)
num_thread int | None Determines how many separate conversations or tasks the machine can handle at the same time. (Example: 8)
repeat_last_n int | None How much of the last part of the conversation to try not to repeat. (Example: 64)
repeat_penalty float | None A nudge to encourage the machine to come up with something new if it starts repeating itself. (Example: 1.1)
temperature float | None Controls how “wild” or “safe” the machine’s responses are. (Example: 0.7)
seed int | None Sets up a starting point for generating responses. (Example: 42)
stop str | None Tells the machine when to stop talking, based on certain cues or keywords. (Example: "AI assistant:")
tfs_z float | None Aims to reduce randomness in the machine’s responses. (Example: 2.0)
num_predict int | None Limits how much the machine can say in one go. (Example: 128)
top_k int | None Limits the machine’s word choices to the top contenders. (Example: 40)
top_p float | None Works with top_k to fine-tune the variety of the machine’s responses. (Example: 0.9)
Return type: ModelSettings

➡️ reset()

Reset all properties to their initial values
Return type: None

➡️ get_settings()

Returns a dictionary of all the settings and their current values, excluding None values.
Return type: dict