Building a Playground with AI SDK

Building a Chatbot

You can refer to the Typescript Core SDK examples to learn how to build a chatbot. Of course, you can also accomplish this by directly calling various LLM APIs, which we won't elaborate on here.

Creating a New Sandbox

You should create a new sandbox on the server side and obtain the sandbox ID.

use lybic_sdk_rs::{Client, types::CreateSandboxDto};

const ORG_ID: &str = "lybic-sdk-rust-example";

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    // Create a client
    let client = Client::new("https://api.lybic.cn");

    // Create a sandbox
    let sandbox = client
        .create_sandbox(
            ORG_ID,
            &CreateSandboxDto {
                max_life_seconds: Some(3600.0),
                name: ORG_ID.to_string(),
                project_id: None,
                shape: "<your-sandbox-shape>".to_string(),
            },
        )
        .await?
        .into_inner();

    println!("Sandbox created: {:?}", sandbox);

    Ok(())
}

Real-time Preview of Sandbox Screen

Getting Started with Lybic UI SDK

Check out the Lybic UI SDK to learn more about how to connect the preview interface to the sandbox.

Taking a Screenshot of the Sandbox

let result = client.preview_sandbox(ORG_ID, &sandbox.id).await?.into_inner();

if let Some(screen) = result.screen_shot {
    // Process the screenshot
}

You may also need to download the screenshot and convert it to a Base64 data URL, as some LLM APIs may not support HTTPS URLs.

Parsing LLM Responses and Executing Actions

The LLM may respond with actions such as moving or clicking the mouse, keyboard input, etc. You can parse the LLM's output and convert it to Lybic's action space to have the sandbox execute these operations:

let result = client.execute_computer_use_action(ORG_ID, &sandbox.id, &ComputerUseActionDto {
    action: ComputerUseActionDtoAction {
        // The action you want to execute
    },
    include_cursor_position: true,
    include_screen_shot: true,
}).await?.into_inner();

if let Some(screen) = result.screen_shot {
    // Contains the screenshot after executing the action
}

Conclusion

The Lybic SDK primarily encapsulates common sandbox operations. You can use these APIs to operate a computer, simulate user actions, and integrate with LLM to provide AI with the ability to operate the sandbox.