
Tools Reference
This is the complete list of built-in tools available to your Digital Twin. The AI automatically selects which tool(s) to use based on the conversation context.Some tools require additional configuration (e.g.,
call_canvas requires Canvas LMS integration, search_kaltura requires Kaltura credentials). Tools without required configuration are automatically disabled.Web & Research Tools
get_browser
get_browser
Searches trusted sites with Serp APIsUse for up-to-date information on broad topics when the exact URL is unknown:
- Current events, weather, stock tickers, sports scores
- General web research not covered by specialized tools
- Information you’re unfamiliar with
- “Search for the latest news on climate change.”
- “What’s the current weather in Tokyo?”
- “Find the most popular tourist attractions in Paris.”
Supports real-time voice conversations.
read_url
read_url
Read content at URLs on the internetAccess, retrieve, summarize, or save content from a known web page URL. Provides download links in Markdown format for public files.Examples:
- “Read the content from the Wikipedia page on the Roman Empire.”
- “Summarize the main points from https://example.com/article”
- “Save the content from this GitHub README to my files.”
Supports real-time voice conversations.
get_scholar
get_scholar
Searches Google ScholarSearch for scholarly articles and publications from peer-reviewed sources including NIH, NIM, PubMed, DOI, and more.Examples:
- “Find scholarly articles on artificial intelligence in healthcare.”
- “Search for recent studies on the effects of social media on mental health.”
- “What does the research say about intermittent fasting?”
get_patent
get_patent
Searches Google PatentsSearch for information about patents and inventions worldwide.Examples:
- “Who owns the patent for the smartphone?”
- “Find patents related to virtual reality technology filed in the last five years.”
- “Search for patent ID US10123456.”
get_clinicaltrial
get_clinicaltrial
Searches ClinicalTrials.govSearch clinical research studies worldwide from the US National Library of Medicine, including purpose, recruitment status, eligibility criteria, and locations.Examples:
- “Search for clinical trials related to diabetes treatment.”
- “Find clinical studies on mindfulness meditation for stress reduction.”
- “What’s the status of trial NCT06310070?”
get_wolframalpha
get_wolframalpha
Searches WolframAlpha for computations and visualizationsFactual lookups, quantitative analysis, symbolic/numeric computation (integrals, sequences, financial math), and visualizations across science, math, geography, economics, weather, population, music, and space. Also supports bio/chem/finance tasks (e.g., DNA→amino acids translation, Taylor series, compound interest calculations).Response Styles:
short— Terse answer only (minimal prose)image— Return response as an image (best for complex graphs)full— Detailed steps/derivations with richer context (default)spoken— Phrasing suitable for voice narration
- “What’s the nutritional content of an apple?”
- “Calculate the integral of sin(x) from 0 to π.”
- “Weather in NYC on July 4, 1776.”
- “California population from 1900 to 2020.”
- “Future value of $1,000 at 5% for 10 years.”
- “Translate this DNA sequence to amino acids.”
Supports real-time voice conversations.
Media & Content Tools
get_video
get_video
Searches Google YouTubeSearch for video content on specific topics from YouTube and other sources.Examples:
- “Find an educational video about the solar system.”
- “Show me a tutorial on how to bake sourdough bread.”
- “Search for analysis videos on current economic events.”
Supports real-time voice conversations.
transcribe_youtube
transcribe_youtube
Transcribe a video from YouTubeTranscribe video content when the full YouTube URL is known. Does not work for non-YouTube videos or audio files.Parameters:
- URL: The YouTube video URL (e.g.,
https://www.youtube.com/watch?v=...orhttps://youtu.be/...) - Save Result: Set to
trueto save/download the transcript to your files
- “Transcribe the video at https://www.youtube.com/watch?v=Z8T149nP5cg”
- “Summarize the YouTube video at https://youtu.be/8nBFqZppIF0”
- “Download the transcript from this YouTube video.”
transcribe_audio
transcribe_audio
Transcribe an audio file from a URLTranscribe audio files from a URL. Does not work for YouTube videos. The AI checks if transcripts are available in your files before using this tool.Supported Formats: mp3, mp4, mpeg, mpga, m4a, wav, webmExamples:
- “Transcribe the audio file at https://example.com/recording.mp3”
- “Convert this podcast episode to text.”
- “Transcribe the audio at https://upload.wikimedia.org/wikipedia/en/4/48/Bernard_Lord_at_Ecole_Odyssee_inauguration.ogg”
search_kaltura
search_kaltura
Search Kaltura for instructional videosSearch or list instructional videos on the Kaltura video streaming platform. Supports filtering by author and extracting transcripts.Parameters:
- Topic: The topic to search (leave empty for all videos)
- Author: Filter by author’s user ID, name, or email (comma-separated)
- Extract Transcripts: Set to
trueto include video transcripts - Save Results: Set to
trueto download search results
- “Find an Algebra video on Kaltura.”
- “What are my videos on Kaltura?”
- “Download the transcripts of a video about genome sequencing from Kaltura.”
- “Find videos from [email protected] on Kaltura.”
Supports real-time voice conversations.
search_image
search_image
Search for images using Unsplash.orgSearch for free images in the Unsplash database. Returns images with courtesy notices for attribution.Parameters:
- Description: A short description of the image to search
- Orientation:
landscape,squarish, orportrait(default) - Count: Number of images to return (default: 1)
- “Find a sunset over the mountains.”
- “Search for 3 landscape photos of the ocean.”
- “Find a portrait-oriented image of a forest.”
Supports real-time voice conversations.
search_sound
search_sound
Search for sounds using Freesound.orgSearch for free sounds in the Freesound database. For sound effects only, not music.Parameters:
- Search Terms: Space-separated terms, or phrase in double quotes. Use
+for mandatory and-for prohibited terms (e.g.,"thunder -rain") - Count: Number of sounds to return (default: 1)
- “Find a sound of whales talking to each other.”
- “Search for thunder and rain sounds.”
- “Find 5 different bird call sounds.”
Supports real-time voice conversations.
get_map
get_map
Searches Google MapsSearch for places, points of interest, geographical information, or generate maps of specific areas.Examples:
- “Show me a map of downtown Paris.”
- “Find Italian restaurants in Downtown Seattle.”
- “What are the points of interest near Times Square?”
get_location_for_gps_coordinates
get_location_for_gps_coordinates
Get address from GPS coordinatesObtain the address of a place based on latitude and longitude coordinates.Examples:
- “What’s the address at coordinates 40.7128, -74.0060?”
- “What city is at latitude -21.2863697 and longitude 55.4108081?”
Supports real-time voice conversations.
Image Tools
generate_image
generate_image
Generate or edit images using AICreate new images or edit existing ones using the configured Image Generation Model (e.g., DALL-E, Amazon Titan). Prompts are automatically translated to English. Supports generating multiple variations from the same prompt.Parameters:
- Prompt: Detailed description of the image to create
- Source URLs: Optional space-delimited list of image URLs to edit/extend (JPEG, PNG, WebP only)
- Count: Number of image variations to generate (default: 1)
- No politicians or public figures
- No copyrighted characters (original descriptions only)
- Supports editing JPEG, PNG, and WebP files
- “Generate an image of a serene lake at sunset with mountains.”
- “Create a 3D picture of DNA proteins with bright colors.”
- “Edit this image to add a rainbow in the sky.”
- “Generate 3 variations of a futuristic cityscape.”
Supports real-time voice conversations.
analyze_image
analyze_image
Analyze and describe content in imagesIdentify, describe, extract text, count elements, determine emotions, historical context, cultural significance, and accessibility options in images.Examples:
- “Analyze this image of the Eiffel Tower for historical context.”
- “Extract the text contained in this screenshot.”
- “Describe the species of birds in this photo.”
- “Count the number of people in this image.”
Document & File Tools
call_rag
call_rag
Retrieve content from user files via semantic searchSearch for content relevant to the conversation contained in user files using Euclidean kNN (k-Nearest Neighbor) search. Results are ordered by relevance score.Examples:
- “Search my documents for information about project budgets.”
- “Find content in my files related to marketing strategies.”
- “What do my uploaded documents say about compliance requirements?”
Supports real-time voice conversations.
get_user_files
get_user_files
Retrieve user files with filteringAccess the user’s document library with metadata including summaries, types, dates, dimensions, and authors. Filter by file type or keyword search.Parameters:
- Type Filter: Filter by
image,audio,video, ordocument(for all other files) - Keyword: Search within file names and document summaries (exact match)
- Include Content: Set to
trueto include file content in response
- “List all my uploaded documents.”
- “Show me my image files.”
- “Find files with ‘report’ in the name.”
- “What audio files do I have?”
- “Show me the contents of my research documents.”
Supports real-time voice conversations.
save_to_file
save_to_file
Save text content to user’s document collectionSave or append text content to files. Supports text-based formats only (MD, TXT, JSON, CSV, HTML, JS, CSS, XML, YAML, TEX, TSV). Binary formats (PDF, DOCX) are not supported.Parameters:
- Content: The text content to save (required)
- Filename: File name without extension. Auto-generated if not specified using format
{subject}_{YYYY_MM_DD_HH_MM} - Extension: File extension without leading dot (default:
txt) - Overwrite: Set to
trueto replace existing file,falseto append (default)
- “Save this conversation summary to a file called meeting_notes.”
- “Create a new markdown file with this content.”
- “Append these notes to my existing research.txt file.”
- “Save this code snippet as a JavaScript file.”
Supports real-time voice conversations.
Memory Tools
set_in_memory
set_in_memory
Save parameters to user or shared memoryStore or clear parameters in memory. Each parameter includes a namespace (for categorization), name, value, and description. Use namespaces to organize parameters and avoid key name conflicts. Shared memory is accessible by all users of the Digital Twin.Parameters:
- Parameters Array: JSON array with
key_name,key_value,key_description, andkey_namespaceproperties - Shared Memory: Set to
trueto save in shared memory (accessible by all users) - Clear Memory: Set to
trueto remove all parameters from memory
- “Remember that my birthday is December 30th, 1972.”
- “Remember I prefer short and concise responses.”
- “Save this copyright notice in shared memory: © 2025, Praxis-AI, Inc”
- “Clear all my stored preferences from memory.”
Supports real-time voice conversations.
get_from_memory
get_from_memory
Retrieve parameters from memoryRetrieve a list of parameters from user or shared memory. Returns CSV-formatted data including namespaces, names, values, descriptions, and creation dates. Search is case-insensitive and token-based.Parameters:
- Search Term: Search by key name, description, or namespace (leave empty for all parameters)
- Shared Memory: Set to
trueto search in shared memory instead of user memory
- “What’s my morning routine according to memory?”
- “What preferences have I stored?”
- “Show me all parameters in shared memory.”
- “What do you remember about my personal data?”
Supports real-time voice conversations.
Visualization Tools
create_chart_2d
create_chart_2d
Create 2D charts for data visualizationGenerate charts for visualizing single or multiple data series with categories on the x-axis. Each data series should include values corresponding to the specified categories.Parameters:
- Title: Descriptive title for the chart
- Categories: JSON array of x-axis category names (e.g.,
['January', 'February', 'March']) - Data Series: JSON array of data series objects with
nameanddataproperties (e.g.,[{name: 'Sales', data: [10, 8, 11]}])
- “Create a chart showing sales data from January to April.”
- “Generate a chart comparing monthly rainfall to temperature.”
- “Visualize this quarterly revenue data as a bar chart.”
LMS Integration Tools
call_canvas
call_canvas
Interact with Canvas LMSAccess course material (syllabus, pages, office hours), enrolled users, assignments, submissions, grades, announcements, and discussion topics from Canvas LMS via the Canvas API.Parameters:
- Method: HTTP request method (GET, POST, PUT, DELETE)
- Endpoint: Canvas API endpoint URL (without domain)
- Body: Request body for POST/PUT requests (form-urlencoded or JSON)
- Faculty Context: Set to
truefor faculty/admin actions (grading, curating content)
- List and retrieve course content, syllabus, and pages
- View and submit assignments
- Read and post discussion entries
- Access grades and submission feedback
- Smart search via
/api/v1/courses/:course_id/smartsearch
- “What are my assignments due this week?”
- “Show me the course syllabus.”
- “List discussion topics in my course.”
- “What’s my grade on the midterm exam?”
- “Post a comment on my assignment submission.”
Supports real-time voice conversations.
search_canvas
search_canvas
Search Canvas for course materialFallback search tool when
call_canvas is not available. Searches course material including syllabus, pages, content, assignments, grades, office hours, and announcements.Examples:- “When is my homework due this week for Intro to Marketing?”
- “Search for office hours in my course.”
- “Find content about Chapter 5 in my course.”
Bioinformatics Tools
search_blast
search_blast
BLAST sequence similarity searchFind regions of similarity between biological sequences using NCBI BLAST (nucleotide/protein comparison with statistical significance). Note: Queries may take several minutes.Parameters:
- Program: megablast, blastn, blastp, rpsblast, blastx, tblastn, or tblastx (required)
- Database: Target database for comparison (required)
- Sequence: Nucleotide or protein sequence to search (required)
- Save Results: Set to
trueto download results
- All programs: core_nt, refseq_select_rna, refseq_rna, nt, Whole_Genome_Shotgun_contigs, est, sra, patnt, pdbnt
- Protein (blastp, blastx): nr, refseq_select_prot, refseq_protein, swissprot, pataa, pdb
- Nucleotide (blastn): rRNA_typestrains/16S_ribosomal_RNA, nt_euk, nt_prok, nt_viruses
- “Search blastp for this protein sequence: MKVKVLSLLVP…”
- “Run a megablast search against the nt database.”
- “Compare this DNA sequence to RefSeq genomes.”
Communication Tools
send_email
send_email
Send an emailSend emails with specified recipients, subject line, and HTML-formatted message body. Supports multiple recipients and responsive HTML formatting.Parameters:
The AI will request confirmation before sending unless instructed otherwise in your Digital Twin’s system prompt.
- Recipients: Email address(es), comma-separated for multiple recipients (required)
- Subject: Email subject line (required)
- Body: HTML-formatted message with inline CSS and responsive tags like
<p>,<br>,<strong>(required)
- “Send an email to [email protected] about the meeting tomorrow.”
- “Email the team a summary of today’s discussion.”
- “Send this report to [email protected] and [email protected].”
Supports real-time voice conversations.
Assistant Tools
get_assistants
get_assistants
List available assistantsRetrieve a filtered or complete list of conversational assistants accessible to the current user. Returns assistant IDs and names.Examples:
- “What assistants are available?”
- “Find assistants with ‘math’ in the name.”
- “List all my assistants.”
Available in real-time voice mode only.
start_assistant
start_assistant
Activate an assistantInitialize and activate a specific conversational assistant, loading its specialized instructions into the current session.Examples:
- “Start the Math Tutor assistant.”
- “Switch to the Writing Coach.”
- “Activate the Code Review assistant.”
Available in real-time voice mode only.
API Integration Tools
call_rest
call_rest
Execute REST API callsExecute authenticated REST API calls to specific service endpoints. Use only when: (1) explicitly instructed by name, (2) the endpoint requires authentication other tools can’t provide, or (3) the service is internal/proprietary.Parameters:
- Bearer Token: Authorization token (leave empty if no auth required)
- URL: Complete REST endpoint URL with all variables substituted
- Method: HTTP method (GET, POST, PUT, DELETE)
- Body: Request body in JSON format for POST/PUT/DELETE
- Query Params: URL-encoded query string for GET/DELETE (RFC 3986)
- Content Type: Header value (default:
application/json)
- “Call the attendance API endpoint with my bearer token.”
- “POST this data to the internal reporting service.”
- “Fetch data from our custom REST endpoint.”
Supports real-time voice conversations.
Tool Availability
Tools are automatically enabled or disabled based on your Digital Twin’s configuration:| Tool | Requires | Status |
|---|---|---|
call_canvas, search_canvas | Canvas LMS integration | Active |
search_kaltura | Kaltura credentials | Active |
search_blast | Institutional account | Active |
search_drive | Google Drive integration | Inactive (coming soon) |
Real-Time Voice Support
Tools marked withSupports real-time voice conversations.
Available in real-time voice mode only.