Standalone analytics provider and realtime dashboard designed for self-hosting.

Overview

Stats

Stats is a high-performance, standalone analytics provider designed for self-hosting, enabling the collection and viewing of event data from websites, apps, and more through a web API. It comes with a real-time dashboard and is ideal as a minimal and privacy-preserving analytics solution for small to medium traffic services.

Current status: it works, but it's not finished.

Stats dashboard on iPad

Key-features

  • Real-time analytics and dashboard
  • Lightweight and efficient, with minimal resource usage
  • Easy integration with websites, mobile apps, and other platforms

Getting started

To get started with Stats, you'll need to compile an executable for your server, run it, and host it. While this guide focuses on using Nginx as a reverse proxy, Stats is versatile enough to be deployed in various environments, including Docker containers, for enhanced portability and scalability.

Run in development-mode

RUST_LOG=debug cargo run

Build production release

cargo build --release

This will create the executable file you need in the /target/release/ folder

Embed events collector
You use this to automatically collect pageviews or other events triggered by calling stats_collect('event_name', 'optinal_url_override') from javascript once the script below is initialized.

When you add this script to a new domain, you must add them to the CORS_DOMAINS list on the backend so the server can receive data from them.

<script>
  // Stats analytics
  var head = document.head || document.getElementsByTagName("head")[0];
  var script = document.createElement("script");
  script.setAttribute("src", "http://localhost:5775/stats.js"); // REPLACE WITH ACTUAL URL
  script.setAttribute("onload", () => window.collectStats());
  script.setAttribute("type", "text/javascript");
  script.setAttribute("charset", "utf8");
  script.setAttribute("async", "");
  head.appendChild(script);
</script>

Setup

Minimum set of folders & files required to run this application.

stats/
├── data/
│ ├── GeoLite2-City.mmdb
│ ├── cities5000.txt 
│ └── stats.sqlite 
├── ui/
├── stats // copy executable from target/release/stats
└── .env

Configuration

These options must be defined in a .env file before starting the server.

Variable Default Summary
APP_URL http://localhost:5775 Full domain you are hosting this service on
SERVICE_PORT 5775 Port you want the service to be hosted from
DATABASE_URL /data/stats.sqlite Path to .sqlite file to use as database.
CORS_DOMAINS http://localhost:5775,https://udara.io Comma-separated list of allowed domains. The service will only accept analytics events from these domains.
PROCESSING_BATCH_SIZE 500 Max limit for events buffer used to queue and batch analytics events for processing. When the limit is hit, new events are dropped until items are processed from the queue.
Comments
  • repeated calls

    repeated calls

    Maybe it is a possibility to show the repeated calls. This way you can easily see that there are duplicated calls. (like they show in debug toolbar)

    opened by JostCrow 9
  • Added an

    Added an "elapsed" and "query count" log threshold optional setting and documented the settings in readme

    This is a very useful tool to debug django db performance.. but data can get cluttered between tons of small and meaningless requests. this allows the middleware user to selected the query elapsed time and the minimal query count that he wants the middleware to log. Enabling the user to focus on bottlenecks and problems.

    Without explicitly changing the settings the current behavior isn't changed in any way

    opened by alonisser 3
  • Feature/response header count and setting reload

    Feature/response header count and setting reload

    This PR adds functionality to return a custom response header with total request & response query count. We plan to use this functionality in our integration test suite, to ensure all endpoints are under the allowed count.

    Additionally, when attempting to override the settings to ensure the header was being removed, I found the need to ensure the settings are reloaded, so I've cleaned up the settings application and wired them up to the Django setting changed signal.

    opened by LifeCoder45 3
  • add IGNORE_SQL_PATTERNS option to ignore the specified sql query

    add IGNORE_SQL_PATTERNS option to ignore the specified sql query

    Now django-querycount statistic all sql querys, this pr makes django-querycount can filter sql query by IGNORE_SQL_PATTERNS.

    opened by GitFree 1
  • Update README, add fix typo statement in settings config

    Update README, add fix typo statement in settings config

    Fix simple typo in README

    opened by essanpupil 1
  • Support Django 1.10 middleware

    Support Django 1.10 middleware

    This adds support for Django 1.10-style middleware: https://docs.djangoproject.com/en/1.10/topics/http/middleware/#upgrading-pre-django-1-10-style-middleware

    opened by ryankask 1
  • Changed settings name

    Changed settings name

    opened by pyprism 1
  • Disable colors when not supported (or explicitely disabled)

    Disable colors when not supported (or explicitely disabled)

    On Windows, using powershell, the output looks bad as colors are not supported :

    http://127.0.0.1:8000/admin/jsi18n/
    ?[0m|------|-----------|----------|----------|----------|------------|
    | Type | Database  |   Reads  |  Writes  |  Totals  | Duplicates |
    |------|-----------|----------|----------|----------|------------|
    ?[32m| RESP |  default  |    2     |    0     |    2     |     1      |
    ?[0m|------|-----------|----------|----------|----------|------------|
    ?[32mTotal queries: 2 in 0.0160s
    

    Colors should be disabled when not supported or disabled. I believe Django has all the logic needed in django\core\management\color.py

    opened by olivierdalang 0
  • "Repeated 1 times"

    It prints this:

    web_1            | Repeated 1 times.
    web_1            | SELECT `django_session`.`session_key`,
    web_1            | `django_session`.`session_data`, `django_session`.`expire_date` FROM
    web_1            | `django_session` WHERE (`django_session`.`session_key` =
    web_1            | 'cln2uzh2qk21qqb6sfg7nvsrqwg2crbm' AND `django_session`.`expire_date`
    web_1            | > '2019-09-20 12:53:39.352699')
    

    But I think it actually ever happens 1 time for that request. So, the logic for this inclusion should be something like [query for queries if query.times > 1].

    opened by peterbe 4
  • How to view the total time of repeated sql queries

    How to view the total time of repeated sql queries

    Can we show the total time take for the repeated calls individually

    For example: presently it shows Repeated 2 times.

    Can we have Repeated 2 times. [0.023]

    opened by sant527 0
  • Using sql parse and pygments to show sql formatted nicely

    Using sql parse and pygments to show sql formatted nicely

    I wanted the sql to be formatted so i have tried the below code for that. Can you incorporate sqlparse also if possible

    def _duplicate_queries(self, output):
            """Appends the most common duplicate queries to the given output."""
            if QC_SETTINGS['DISPLAY_DUPLICATES']:
                for query, count in self.queries.most_common(QC_SETTINGS['DISPLAY_DUPLICATES']):
                    lines = '\nRepeated {0} times.'.format(count)
                    #lines += wrap(query)
                    #lines = "\n".join(lines) + "\n"
                    lines += "\n" + self._str_query(query) + "\n"
                    output += self._colorize(lines, count)
            return output
    
        def _str_query(self,sql):
    
            # Check if Pygments is available for coloring
            try:
                import pygments
                from pygments.lexers import SqlLexer
                from pygments.formatters import TerminalTrueColorFormatter
            except ImportError:
                pygments = None
            # Check if sqlparse is available for indentation
            try:
                import sqlparse
            except ImportError:
                sqlparse = None
            # Remove leading and trailing whitespaces
            if sqlparse:
                # Indent the SQL query
                sql = sqlparse.format(sql, reindent=True)
            if pygments:
                # Highlight the SQL query
                sql = pygments.highlight(
                    sql,
                    SqlLexer(),
                    #TerminalTrueColorFormatter(style='monokai')
                    TerminalTrueColorFormatter()
                )
    
            return sql
    
    opened by sant527 1
  • Whitelist request patterns instead of blacklisting them

    Whitelist request patterns instead of blacklisting them

    I often debug a single endpoint with a unique request pattern. To be able to only see that in the console I have to blacklist all other request patterns that my application may produce during debugging. That can be quite annoying. It would be helpful to have a separate whitelist setting like INCLUDE_REQUEST_PATTERNS where I just state what I want to be printed to the console and everything else is silenced. Of course the User would have to decide whether to blacklist or whitelist, as using both does not make much sense.

    I can do a PR if you want, let me know if it's something worth putting time into.

    opened by cb109 1
  • Replace <some_id>=<some_number> with <some_id>=#number# in WHERE clause.

    Replace = with =#number# in WHERE clause.

    This will many duplicates that only vary by <some_number>, indicating an N+1 condition.

    opened by wsmith323 0
  • print SQL commands

    print SQL commands

    Hey, is it possible to print SQL commands like here (screenshot from README)?

    image

    I have default settings and get only queries number and time without SQL

    opened by Marpop 1
  • Usage without request and response

    Usage without request and response

    Hi, thanks for this awesome package. I really enjoy using it.

    But I have a question about how to use this package without a request instance? Here are some examples where I need this:

    1. celery tasks, in DEBUG I want them to print the query count in the same manner middleware does
    2. telegram bots, we do develop some telegram bots on top of django. telegram bots serve as clients which does not have any django's request instance. But I still want this app to work

    Thanks!

    opened by sobolevn 3
Owner
Udara Jay
Make cool things. Be nice.
Udara Jay
🦑 GitHub Dashboard

ghboard ?? GitHub dashboard written in Rust ?? , made using Dioxus SSR ?? , hosted in Shuttle ?? and powered by the GitHub GraphQL API ?? . [ ⚠️ Work

Marc Espín 24 Jul 11, 2023
Node.js Project Template for hosting on shuttle.rs

Node.js on shuttle.rs Template This Template is a starting point for creating a Node.js project on the shuttle.rs. It is a simple discord bot template

Arnav Kumar 7 Apr 18, 2024
A site for hosting (Japanese) subtitles

jimaku (字幕) jimaku is a simple site dedicated to hosting Japanese subtitles of anime or other Japanese content. It's the spiritual successor of kitsun

Danny 47 Jul 21, 2024
siamstr.com Nostr Address Provider written in Rust.

Siamstr Nostr Address Provider Running project Rust Toolchain You'll need to use the nightly Rust toolchain, and install the wasm32-unknown-unknown ta

Vaz. 5 Nov 27, 2023
A standalone code editor with syntax highlighting and themes.

CodeEditor A standalone code (and text) editor for people like me who write their own user interfaces utilizing crates like pixels. CodeEditor renders

Markus Moenig 8 Nov 25, 2022
A standalone Command Line Interface debugging tool for The Witcher 3 written in Rust

A standalone Command Line Interface debugging tool for The Witcher 3 written in Rust. This tool is intended for Witcher 3 modders who make mainly scri

Przemysław Cedro 5 Apr 20, 2022
📦 Distribute Roblox games as standalone executables -- No existing client necessary. 🚧

?? Packer Distribute Roblox games as standalone executables. ?? Packer is still being worked on. Among many other things, Windows is not currently sup

Brooke Rhodes 16 Dec 20, 2022
A standalone Luau script runner 🌙

Lune ?? A standalone Luau script runner ?? Use the ergonomics and readability of Luau for your shell scripts ?? ⚙️ Installation The preferred way of i

Filip Tibell 13 Feb 2, 2023
Self-contained template system with Handlebars and inline shell scripts

Handlematters Self-contained template system with Handlebars and inline shell scripts Introduction Handlematters is a template system that combines Ha

Keita Urashima 3 Sep 9, 2022
A self-contained, unopinionated, fast and lightweight executable launcher.

Kickoff ?? A self-contained, unopinionated, fast and lightweight executable launcher. Supported Platforms Platform Host Target aarch64-apple-macos-non

Nimbus 18 Oct 27, 2023
Work-in-progress Rust application that converts C++ header-only libraries to single self-contained headers.

unosolo Work-in-progress Rust application that converts C++ header-only libraries to single self-contained headers. Disclaimer This is my first Rust p

Vittorio Romeo 26 Jul 9, 2021
Guardian Self Assessment CLI tool

Guardian Self Assessment CLI tool What? self-assessment is a tool that generates a list of PRs authored and reviewed by you. Why? Assessing oneself is

The Guardian 5 Jul 6, 2022
CLI for self-bootstrapped Python applications

PyApp PyApp is a CLI wrapper for Python applications that bootstrap themselves at runtime. Each application is configured with environment variables a

Ofek Lev 6 May 10, 2023
Temporary elevated access management as a self-hosted service

????☁️ S A T O U N K I Temporary elevated access management as a self-hosted service Overview Satounki is a self-hosted service which brings visibilit

جاد 31 Dec 17, 2023
a simple, non-self-describing data-interchange format.

rust-fr 'rust-fr' (aka rust for real) is a simple, non-self-describing data-interchange format. installation You can use either of these methods. Add

Ayush 4 Feb 28, 2024
Self-Hosted Remote Dev Environment

Lapdev Self-hosted remote development enviroment management with ease Lapdev is a self hosted application that spins up remote development environment

Lapce 1.5k Oct 24, 2024
ratlab is a programming platform designed loosely for hobbyist and masochist to analyse and design stuff and things that transform our world?

ratlab A programming language developed by Quinn Horton and Jay Hunter. ratlab is a programming platform designed loosely for hobbyists and masochists

Jay 10 Sep 4, 2023
A lightweight and high-performance order-book designed to process level 2 and trades data. Available in Rust and Python

ninjabook A lightweight and high-performance order-book implemented in Rust, designed to process level 2 and trades data. Available in Python and Rust

Ninja Quant 134 Jul 22, 2024
Nodium is an easy-to-use data analysis and automation platform built using Rust, designed to be versatile and modular.

Nodium is an easy-to-use data analysis and automation platform built using Rust, designed to be versatile and modular. Nodium aims to provide a user-friendly visual node-based interface for various tasks.

roggen 19 May 2, 2023