🍕 RepoQuery 🍕
A REST service to answer user-queries about public GitHub repositories
🔎 The Project
RepoQuery is an early-beta project, that uses recursive OpenAI function calling paired with semantic search using multi-qa-MiniLM-L6-cos-v1 to index and answer user queries about public GitHub repositories.
📬 Service Endpoints
Note: Since the service returns responses as SSEs, a REST client like Postman is recommended. Download it here. The Postman web client doesn't support requests to
localhost.
1. POST /embed
To generate and store embeddings for a GitHub repository.
Parameters
The parameters are passed as a JSON object in the request body:
owner(string, required): The owner of the repository.name(string, required): The name of the repository.branch(string, required): The name of the branch.
Response
The request is processed by the server and responses are sent as Server-sent events(SSE). The event stream will contain the following events with optional data.
repo-query/src/routes/events.rs
Lines 14 to 20 in f2f415a
Example
curl --location 'localhost:3000/embed' \
--header 'Content-Type: application/json' \
--data '{
"owner": "open-sauced",
"name": "ai",
"branch": "beta"
}'
2. POST /query
To perform a query on the API with a specific question related to a repository.
Parameters
The parameters are passed as a JSON object in the request body:
query(string, required): The question or query you want to ask.repository(object, required): Information about the repository for which you want to get the answer.owner(string, required): The owner of the repository.name(string, required): The name of the repository.branch(string, required): The name of the branch.
Response
The request is processed by the server and responses are sent as Server-sent events(SSE). The event stream will contain the following events with optional data.
repo-query/src/routes/events.rs
Lines 22 to 29 in f2f415a
Example
curl --location 'localhost:3000/query' \
--header 'Content-Type: application/json' \
--data '{
"query": "How is the PR description being generated using AI?",
"repository": {
"owner": "open-sauced",
"name": "ai",
"branch": "beta"
}
}'
3. GET /collection
To check if a repository has been indexed.
Parameters
owner(string, required): The owner of the repository.name(string, required): The name of the repository.branch(string, required): The name of the branch.
Response
This endpoint returns an OK status code if the repository has been indexed by the service.
Example
curl --location 'localhost:3000/embed?owner=open-sauced&name=ai&branch=beta'
🧪 Running Locally
To run the project locally, there are a few prerequisites:
- The Rust toolchain
- The Onnx Runtime. Will be downloaded and installed automatically when building the project.
- Docker to run the QdrantDB instance.
makefor easy automation and development workflow
Once, the above requirements are satisfied, you can run the project like so:
Environment variables
The project requires the following environment variables to be set.
OPENAI_API_KEY. To authenticate requests to OpenAI.
Database setup
Start Docker and run the following commands to spin-up a Docker container with a QdrantDB image.
docker pull qdrant/qdrant
docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant
The database dashboard will be accessible at localhost:6333/dashboard, the project communicates with the DB on port 6334.
Running the project
Run the following command to install the dependencies and run the project on port 3000.
cargo run --release
This command will build and run the project with optimizations enabled(Highly recommended).
🐳 Docker container
The repo-query engine can also be run locally via a docker container and includes all the necessary dependencies.
To build the container tagged as open-sauced-repo-query:latest, run:
make local-image
Then, you can start the repo-query service with:
docker run --env-file ./.env -p 3000:3000 open-sauced-repo-query
There's also a docker-compose.yaml file that can be used to start both qdrant and the repo-query engine together.
To build the image and then start the services, run:
make up
Attributions
https://sbert.net for https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-cos-v1.
@inproceedings{reimers-2020-multilingual-sentence-bert,
title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2020",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/2004.09813",
}
🤝 Contributing
We encourage you to contribute to OpenSauced! Please check out the Contributing guide for guidelines about how to proceed.
We have a commit utility called @open-sauced/conventional-commit that helps you write your commits in a way that is easy to understand and process by others.
🍕 Community
Got Questions? Join the conversation in our Discord.
Find Open Sauced videos and release overviews on our YouTube Channel.
⚖️ LICENSE
MIT © Open Sauced
12 Jul 25, 2023
8 Aug 10, 2023
14 Sep 10, 2022
70 Apr 12, 2023
16 Mar 10, 2024
4.2k Dec 30, 2022
12 Dec 28, 2021
9 Dec 10, 2022
365 Dec 25, 2022
30 Mar 6, 2023
14 May 30, 2023
3 Jul 28, 2023
107 Sep 28, 2022
594 Dec 30, 2022
15 Oct 27, 2021
147 Mar 8, 2023
4 May 15, 2023
5 Jun 20, 2023
6 Aug 7, 2023
5 Sep 5, 2023