TextSynth

A (unofficial) Rust wrapper for the TextSynth API, a free (as of 01/21/22) text synthesization service.

Preparation

You must have an API key in order to use this library.

In order to get an API key, you must create an account at TextSynth.

After signing up and verifying your email address, you will be given an API key.

Basic Usage

The most commonly used types and traits are in the prelude. Import them first:

use textsynth::prelude::*;

We now need to create a TextSynth instance. This is where we can create engines, explained further down below. Pass the API key you got from the preparations to TextSynth::new.

"); let textsynth = TextSynth::new(api_key);">
let api_key = String::from("
    
     "
    );
let textsynth = TextSynth::new(api_key);

We now need to create an Engine. An Engine is where the fun stuff takes place. This is where you can complete text and log probabilities.

However, to create an engine, we need to provide a definition, an EngineDefinition. An EngineDefinition is information needed by the TextSynth API to determine which engine should be used. It also determines the maximum amount of tokens one can use in a text completion.

Usually, you can just use a EngineDefinition::GptJ6B to use the GPT-J model. There are other models available, but this one is probably the one you'll use the most.

let engine = textsynth.engine(EngineDefinition::GptJ6B);

Let's get started with the fun stuff.

Log Probabilities

This is logarithm of the probability that a continuation is generated after a context. It can be used to answer questions when only a few answers (such as yes/no) are possible. It can also be used to benchmark the models.

You need to provide two arguments into this function:

A context, the text you wish to predict a continuation for, and
A continuation, a non-empty string that you wish to predict against the context.

let context = String::from("The quick brown fox jumps over the lazy");
let continuation = String::from(" dog");
let continuation = NonEmptyString::new(continuation)?;
let log_probabilities = engine.log_probabilities(context, continuation).await??;

log_probabilities is of type LogProbabilities, which contains the result from the API. It contains multiple fields.

log_probability: This is a logarithm of the probability of generation of continuation preceded by the context. It is always <= 0.
is_greedy: true if continuation would be generated by greedy sampling from continuation.
total_tokens: Indicate the total number of tokens. It is useful to estimate the number of compute resourced used by the request.

println!("log probability = {}", log_probabilities.log_probability());
println!("is greedy = {}", log_probabilities.is_greedy());
println!("total tokens = {}", log_probabilities.total_tokens());

Text Completion

Since we'll only show basic usages here (for more advanced usages see the documentation), we'll only focus on the parts where most users will be interested in.

In order to begin the process of text completion, we need to provide a prompt, which is the input text.

let prompt = String::from("Once upon a time, there was")
let text_completion = engine.text_completion(prompt);

This creates a builder, which contains many method, but we'll only focus on the most common ones.

Immediate Text Completion

Well, "immediate" text completion. You still need to wait for the API to return the result.

This fetches the whole result, without splitting into multiple chunks.

let text_completion = text_completion.now().await??;

The text field contains the generated text itself.

println!("{}", text_completion.text());

Streaming Text Completion

This returns a stream of text completions.

let text_completion = text_completion.stream().await?;

Iterate over it and you can get the text.

// due to `async` limitations, we must iterate like this
while let Some(text_completion) = text_completion.next().await {
    let text_completion = text_completion???;
    print!("{}", text_completion.text());
    io::stdout().flush()?;
}