Cloud native log storage and management for Kubernetes, containerised workloads

Overview

Parseable Parseable

website status contributors commits activity monthly Github stars License Twitter

Live Demo | Website | API Workspace on Postman

Parseable is an open source, cloud native, log storage and management platform.

Parseable helps you ingest high volumes of log data from various sources (Fluent Bit, Logstash etc or directly from applications). Parseable stores log data into highly compressed Parquet file format. With object storage as primary storage for Parseable, you get seamless scale and flexibility for ever growing log data.

Parseable Overview Parseable Overview

Live Demo

Try out Parseable server with our demo instance.

  1. Post log data to default log stream frontend
curl --location --request POST 'https://demo.parseable.io/api/v1/logstream/frontend' \
--header 'X-P-META-meta1: value1' \
--header 'X-P-META-meta2: value2' \
--header 'X-P-TAGS-tag1: value1' \
--header 'X-P-TAGS-tag2: value2' \
--header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ==' \
--header 'Content-Type: application/json' \
--data-raw '[
    {
        "id": 4,
        "host":"153.10.110.81", 
        "user-identifier":"-", 
        "datetime":"24/Jun/2022:14:12:15 +0000", 
        "method": "DELETE", 
        "request": "/virtual/drive", 
        "protocol":"HTTP/2.0", 
        "status":500, 
        "bytes":21969, 
        "referer": "http://www.google.com/"
    }
]'
  1. Then access the Parseable dashboard to verify the log data is present
URL https://demo.parseable.io
Username parseable
Password parseable

For complete Parseable API documentation, refer to Parseable API workspace on Postman.

πŸ“ Parseable is a work in progress. Features will evolve over time and there may be breaking changes between releases. Please give us your feedback in Slack, or Issues.
❗ Please do not store any sensitive data on this server as the data is openly accessible. We'll delete the data on this server periodically.

Why Parseable

  • Log data compressed and stored in Parquet - columnar, open data format, designed for analytics.
  • Query logs via Parseable or bring your own analytics platform.
  • Stateless design with object storage (S3, MinIO) as primary storage. Allowing cost effective scale and flexibility.
  • Own your log data on your object storage buckets.
  • SDK less, simple REST API calls for log ingestion.
  • Indexing free design.

Features

  • Cloud native design, get started with Pod logs in minutes.
  • Filter log data on labels.
  • SQL compatible API for querying log data.
  • Intuitive dashboard to parse and query the log data.
  • Bring your own analytics platform for deeper analysis of log data.

Contributing

Refer to the contributing guide here.

License

Copyright (c) 2022 Parseable, Inc.

Licensed under the GNU Affero General Public License, Version 3 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.gnu.org/licenses/agpl-3.0.txt

Comments
  • Document FluentBit with Parseable

    Document FluentBit with Parseable

    Parseable is compatible with FluentBit HTTP Output: https://docs.fluentbit.io/manual/pipeline/outputs/http

    We need to document step by step guide on setting up FluentBit with Parseable

    documentation good first issue 
    opened by nitisht 8
  • Improve error message

    Improve error message

    In case a stream is empty, i.e. has no log data yet, the Query API responds with Failed to execute query due to err: missing record batch.

    This is confusing and might not make sense to end user. This error should be changed to say No log entries found in log stream xxx. Also the status code should be 404.

    bug good first issue 
    opened by nitisht 6
  • New log streams not showing up in log streams list

    New log streams not showing up in log streams list

    I was able to create a log stream but it wont show up when I list the streams, even when I have successfully pushed logs into the stream.

    How to reproduce?

    1. Delete log stream (in-case it exists already)
    curl --request DELETE "http://0.0.0.0:8000/api/v1/logstream/testnewlogbug" --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
    
    1. Create a new log stream testnewlogbug
    curl --request PUT "http://0.0.0.0:8000/api/v1/logstream/testnewlogbug" --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
    
    1. List log stream
    curl --request GET "http://0.0.0.0:8000/api/v1/logstream" --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
    

    The stream testnewlogbug does not show up. Double check if testnewlogbug folder is present in s3

    aws --endpoint=https://sgp1.digitaloceanspaces.com --region="sgp1" s3 ls s3://parseable/
    
    opened by 123vivekr 5
  • [Integration] Create stream does not work with s3 object lock enabled.

    [Integration] Create stream does not work with s3 object lock enabled.

    • Based on AWS putObject on S3 documentation, We must pass content-MD5 option to upload objects to S3 with object lock enabled.
    • create stream function does not send the content-MD5 option within the API request sent to AWS Code.
    opened by AhmedmmAli 4
  • Show git semver in banner

    Show git semver in banner

    Closes https://github.com/parseablehq/parseable/issues/146

    Description

    Show git semver in banner


    This PR has:

    • [ ] been tested to ensure log ingestion and log query works.
    • [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
    • [ ] added documentation for new or modified features or behaviors.
    opened by 123vivekr 3
  • Refactor alerts

    Refactor alerts

    Description

    Alerts are stored on metadata map which is behind and rwlock. This is fine but having to hold writer lock for alert check is not ideal. This PR refactors code around alerts implementation. Each alerts rule should be self contained with its own interior mutable state if it requires any. Interior mutablity pattern allows for alert to be resolved with shared reference. An alert can now contain different variant of rules which can be state or stateless.


    This PR has:

    • [ ] been tested to ensure log ingestion and log query works.
    • [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
    • [ ] added documentation for new or modified features or behaviors.
    opened by trueleo 3
  • Set zip version with only deflate feature enabled

    Set zip version with only deflate feature enabled

    Description

    Set zip crate back to stable crates.io version with minimal features enabled. This gets rid of bzip2 as indirect dependency and transitively on bzip2-sys crate. Console releases on GitHub use deflate method for zip, so Parseable build step should only requires this feature.

    opened by trueleo 3
  • [Document] How to send logs to Parseable from a Rust application

    [Document] How to send logs to Parseable from a Rust application

    Parseable website has docs on how to directly send logs from Go, Python, Java, JS etc. Ref: https://www.parseable.io/docs/log-ingestion/applications/go

    We need to add docs on how to directly sends logs from a Rust application.

    Please send the PR to parseablehq/parseable repo and we'll add it to the website.

    documentation good first issue 
    opened by nitisht 3
  • Populate the demo instance with large amounts of data

    Populate the demo instance with large amounts of data

    Hi. It'd be nice for the demo (demo.parseable.io) to be preloaded with large amounts of data to get a good sense of how it functions. I was playing around with it and could only see two events logged in the last 6 months.

    If it helps, here, I've a cron job that generates and populates dummy, but current data every day. A simple approach like that would do.

    bug 
    opened by knadh 3
  • Metadata management flow

    Metadata management flow

    Current STREAM_INFO map may have an issue with parallel writes to the map for the same stream.

    1. We need to implement a locking mechanism that allows exclusive writes to the map.
    2. Additionally Metadata map get and get methods should handle both local and S3. This way we don’t duplicate all the code to fallback on object store everywhere. a. Get Flow: Fetch metadata from local and fallback to object store if not found. b. Set Flow: Set to object store and then try setting it locally.
    enhancement 
    opened by nitisht 3
  • Add support for automatic stream creation

    Add support for automatic stream creation

    In case the user specifies x-p-stream in the header and the stream doesn't exist, server failed with stream doesn't exist error.

    But in highly automated environments like kubernetes, it is important to dynamically create the stream so that Parseable is able to ingest the log data quickly without too much user involvement.

    This PR has:

    • [x] been tested to ensure log ingestion and log query works.
    • [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
    • [ ] added documentation for new or modified features or behaviours.
    opened by nitisht 2
  • PGO + Bolt

    PGO + Bolt

    parseable right now does not support building with more advanced optimization techniques like PGO and BOLT. This tooling has an increasing adoption in the community as a tool to additionally optimize programs. With this tooling, there is a huge chance to gain even more performance "for free".

    Here I suggest considering an option at least to play with LTO ("fat" version) + PGO + Bolt pipeline (or any combination of them) and test, does it give a performance to the project or not. If yes, would be awesome to have prebuilt binaries with more advanced optimization from the scratch. Also, for the users will be helpful to have the ability to tweak manually their own binaries to their own workloads with the integrated into the build scripts functionality.

    Since the project is quite small I do not expect (at least yet) significant improvement from the PGO right now. But at least a good option to try :)

    Also, there are some caveats to consider like:

    • Significantly increased build times
    • BOLT could be still unstable (or even broken) on some architectures

    Links:

    • Rust experience with LTO + PGO + BOLT: https://kobzol.github.io/rust/rustc/2022/10/27/speeding-rustc-without-changing-its-code.html
    enhancement help wanted 
    opened by zamazan4ik 1
  • [Integration] Develop Parseable Fluent Bit output plugin

    [Integration] Develop Parseable Fluent Bit output plugin

    This plugin is required to make the Fluent Bit + Parseable configuration easier to setup and add new applications for logging.

    Ref: https://www.cncf.io/blog/2022/05/04/how-to-write-a-fluent-bit-plugin/ Ref: https://github.com/fluent/fluent-bit/blob/master/plugins/out_loki/loki.c

    opened by nitisht 0
  • [feature] Support schema evolution in a given stream

    [feature] Support schema evolution in a given stream

    As applications evolve, the log events they generate evolve as well. It would be very useful for Parseable server allow schema evolution of a given stream as source applications evolve. This is also reported a as a common issue with ES.

    This change would

    • further reduce the burden on the user/operator to manage schema
    • ensure application's entire life cycle log event data resides on a single Parseable stream, allowing easier analysis
    • remove fragmentation of log data across streams
    enhancement 
    opened by nitisht 1
Releases(v0.0.8)
  • v0.0.8(Dec 13, 2022)

  • v0.0.7(Dec 9, 2022)

    What's Changed

    • Readme changes for readability by @nitisht in https://github.com/parseablehq/parseable/pull/193
    • refactor readme further by @nitisht in https://github.com/parseablehq/parseable/pull/194
    • Show git semver in banner by @123vivekr in https://github.com/parseablehq/parseable/pull/197
    • increase payload to 10Mb by @AdheipSingh in https://github.com/parseablehq/parseable/pull/198
    • Rust fixes as per vscode by @AdheipSingh in https://github.com/parseablehq/parseable/pull/199
    • Remove duplicated code for processing event by @trueleo in https://github.com/parseablehq/parseable/pull/201
    • Change local cache directory structure by @trueleo in https://github.com/parseablehq/parseable/pull/202
    • Re-implement support for query on local cache by @trueleo in https://github.com/parseablehq/parseable/pull/204
    • Update the URL for checking for new releases by @nitisht in https://github.com/parseablehq/parseable/pull/205
    • Add timeout by @trueleo in https://github.com/parseablehq/parseable/pull/206
    • Refactor query by @trueleo in https://github.com/parseablehq/parseable/pull/207
    • Update to release 0.0.7 by @nitisht in https://github.com/parseablehq/parseable/pull/208
    • Update to rust 1.65 in Dockerfile by @nitisht in https://github.com/parseablehq/parseable/pull/209

    Full Changelog: https://github.com/parseablehq/parseable/compare/v0.0.6...v0.0.7

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(129 bytes)
    parseable_linux_x86_64(57.76 MB)
    parseable_macos_x86_64(46.40 MB)
  • v0.0.6(Nov 10, 2022)

    What's Changed

    • Bump crates to current latest release by @trueleo in https://github.com/parseablehq/parseable/pull/170
    • Update Dockerfile to work with the new changes in cli format by @nitisht in https://github.com/parseablehq/parseable/pull/171
    • docs: Add syslog-ng log forwarding documentation by @sakti in https://github.com/parseablehq/parseable/pull/173
    • Update helm to latest release, also format & fix Dockerfile by @nitisht in https://github.com/parseablehq/parseable/pull/174
    • Set zip version with only deflate feature enabled by @trueleo in https://github.com/parseablehq/parseable/pull/175
    • Update helm chart for better handling of secrets by @nitisht in https://github.com/parseablehq/parseable/pull/176
    • Fix missing fields in the secret / values.yaml by @nitisht in https://github.com/parseablehq/parseable/pull/177
    • [Fix:DockerBuild] Added musl-dev in compiler stage by @Mayon-Francis in https://github.com/parseablehq/parseable/pull/180
    • Add path validation for cert and key by @trueleo in https://github.com/parseablehq/parseable/pull/181
    • Fix system memory calculation and formatting by @trueleo in https://github.com/parseablehq/parseable/pull/182
    • Refactor alerts by @trueleo in https://github.com/parseablehq/parseable/pull/179
    • Divide alert into submodules by @trueleo in https://github.com/parseablehq/parseable/pull/185
    • Add StringRule for alerts by @trueleo in https://github.com/parseablehq/parseable/pull/186
    • Readme cleanup by @nitisht in https://github.com/parseablehq/parseable/pull/187
    • Readme add links by @nitisht in https://github.com/parseablehq/parseable/pull/188
    • Introduce alert state and timeouts by @trueleo in https://github.com/parseablehq/parseable/pull/189
    • Create a Dockerfile that outputs a static binary during build by @Mayon-Francis in https://github.com/parseablehq/parseable/pull/190
    • Introduce Stats API by @trueleo in https://github.com/parseablehq/parseable/pull/191
    • prepare for new release v0.0.6 by @nitisht in https://github.com/parseablehq/parseable/pull/192

    New Contributors

    • @Mayon-Francis made their first contribution in https://github.com/parseablehq/parseable/pull/180

    Full Changelog: https://github.com/parseablehq/parseable/compare/v0.0.5...v0.0.6

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(129 bytes)
    parseable_linux_x86_64(57.60 MB)
    parseable_macos_x86_64(46.74 MB)
  • v0.0.5(Oct 14, 2022)

    What's Changed

    • refactor: improve dockerfile using latest binary by @adripo in https://github.com/parseablehq/parseable/pull/157
    • Add Dockerfile for main branch build and another for release builds by @nitisht in https://github.com/parseablehq/parseable/pull/158
    • Use kebab-case value placeholders by @trueleo in https://github.com/parseablehq/parseable/pull/159
    • docs : add contributors list by @ProgrammerShri in https://github.com/parseablehq/parseable/pull/161
    • Add demo flag documentation in error message by @sakti in https://github.com/parseablehq/parseable/pull/162
    • Use vergen crate by @trueleo in https://github.com/parseablehq/parseable/pull/164
    • Add proper user agent while checking for update by @nitisht in https://github.com/parseablehq/parseable/pull/166
    • Add a top level server command to Parseable by @nitisht in https://github.com/parseablehq/parseable/pull/168
    • update to next release by @nitisht in https://github.com/parseablehq/parseable/pull/169

    New Contributors

    • @adripo made their first contribution in https://github.com/parseablehq/parseable/pull/157
    • @ProgrammerShri made their first contribution in https://github.com/parseablehq/parseable/pull/161
    • @sakti made their first contribution in https://github.com/parseablehq/parseable/pull/162

    Full Changelog: https://github.com/parseablehq/parseable/compare/v0.0.4...v0.0.5

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(129 bytes)
    parseable_linux_x86_64(57.42 MB)
    parseable_macos_x86_64(49.89 MB)
  • v0.0.4(Oct 7, 2022)

    What's Changed

    • Use register_table for execute on cache by @trueleo in https://github.com/parseablehq/parseable/pull/128
    • Update banner to show latest available version by @trueleo in https://github.com/parseablehq/parseable/pull/131
    • Use rustls instead of openssl by @trueleo in https://github.com/parseablehq/parseable/pull/133
    • Fix search issue in console with new release by @nitisht in https://github.com/parseablehq/parseable/pull/135
    • Error type refactor by @trueleo in https://github.com/parseablehq/parseable/pull/137
    • Bump aws_sdk_s3 version and provide sleep and retry impl for S3 client by @trueleo in https://github.com/parseablehq/parseable/pull/138
    • Switch from structopt to clap by @Celeo in https://github.com/parseablehq/parseable/pull/141
    • Add version number for the current object storage format by @nitisht in https://github.com/parseablehq/parseable/pull/143
    • Lock metadata on first event. by @trueleo in https://github.com/parseablehq/parseable/pull/142
    • Add demo flag for config by @trueleo in https://github.com/parseablehq/parseable/pull/149
    • Create all temporary directories on startup by @trueleo in https://github.com/parseablehq/parseable/pull/153
    • Update to latest console release by @nitisht in https://github.com/parseablehq/parseable/pull/155

    New Contributors

    • @Celeo made their first contribution in https://github.com/parseablehq/parseable/pull/141

    Full Changelog: https://github.com/parseablehq/parseable/compare/v0.0.3...v0.0.4

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(129 bytes)
    parseable_linux_x86_64(61.22 MB)
    parseable_macos_x86_64(49.91 MB)
  • v0.0.3(Sep 20, 2022)

    What's Changed

    • Update Parseable helm chart for next release by @nitisht in https://github.com/parseablehq/parseable/pull/107
    • Change x-p-tags to x-p-tag by @trueleo in https://github.com/parseablehq/parseable/pull/111
    • update readme by @nitisht in https://github.com/parseablehq/parseable/pull/112
    • Update description by @nitisht in https://github.com/parseablehq/parseable/pull/113
    • Update .deepsource.toml by @deepsource-autofix in https://github.com/parseablehq/parseable/pull/119
    • Fix code style and anti patterns reported by DeepSource by @nitisht in https://github.com/parseablehq/parseable/pull/118
    • Update .deepsource.toml by @deepsource-autofix in https://github.com/parseablehq/parseable/pull/120
    • Delegate parquet creation to s3sync instead by @nitisht in https://github.com/parseablehq/parseable/pull/117

    Full Changelog: https://github.com/parseablehq/parseable/compare/v0.0.2...v0.0.3

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(129 bytes)
    parseable_linux_x86_64(60.16 MB)
    parseable_macos_x86_64(49.07 MB)
  • v0.0.2(Sep 11, 2022)

    What's Changed

    • Update Parseable helm chart to v0.0.1 by @nitisht in https://github.com/parseablehq/parseable/pull/89
    • Use Datafusion's re-exported arrow and parquet by @trueleo in https://github.com/parseablehq/parseable/pull/93
    • Use Arrow Schema instead of custom schema type. by @trueleo in https://github.com/parseablehq/parseable/pull/94
    • Update readme with get started section by @nitisht in https://github.com/parseablehq/parseable/pull/95
    • Reorg readme for readability by @nitisht in https://github.com/parseablehq/parseable/pull/96
    • Extend readme section for docker by @nitisht in https://github.com/parseablehq/parseable/pull/97
    • Add k8s installation in readme by @nitisht in https://github.com/parseablehq/parseable/pull/98
    • Config Datafusion query with schema by @trueleo in https://github.com/parseablehq/parseable/pull/99
    • Append arrow directly to local data by @trueleo in https://github.com/parseablehq/parseable/pull/100
    • Delay s3sync polling using tokio sleep by @trueleo in https://github.com/parseablehq/parseable/pull/101
    • Fix cases where num cpu is 1 by @nitisht in https://github.com/parseablehq/parseable/pull/102
    • Set max thread priority for local-sync by @trueleo in https://github.com/parseablehq/parseable/pull/104
    • Add .deepsource.toml by @deepsource-autofix in https://github.com/parseablehq/parseable/pull/105
    • Improve LocalStreamWriter along with error types by @trueleo in https://github.com/parseablehq/parseable/pull/106

    Full Changelog: https://github.com/parseablehq/parseable/compare/v0.0.1...v0.0.2

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(129 bytes)
    parseable_linux_x86_64(69.62 MB)
    parseable_macos_x86_64(49.07 MB)
  • v0.0.1(Sep 2, 2022)

    Changelog

    • add badges to readme by @nitisht in https://github.com/parseablehq/parseable/pull/7
    • disable docker ci workflow temporarily by @nitisht in https://github.com/parseablehq/parseable/pull/8
    • update readme by @nitisht in https://github.com/parseablehq/parseable/pull/9
    • update ui build directory by @nitisht in https://github.com/parseablehq/parseable/pull/10
    • Fix syntax error in logstream put handler by @trueleo in https://github.com/parseablehq/parseable/pull/11
    • Update build directory by @ameen4455 in https://github.com/parseablehq/parseable/pull/12
    • Update build directory by @ameen4455 in https://github.com/parseablehq/parseable/pull/13
    • Move authentication middleware to new scope by @trueleo in https://github.com/parseablehq/parseable/pull/14
    • Move logo and images to docs repo and readme update by @nitisht in https://github.com/parseablehq/parseable/pull/15
    • remove localhost calls from ui by @nitisht in https://github.com/parseablehq/parseable/pull/16
    • Fix auth issue by @ameen4455 in https://github.com/parseablehq/parseable/pull/19
    • update link by @nitisht in https://github.com/parseablehq/parseable/pull/21
    • helm chart for parseable by @nitisht in https://github.com/parseablehq/parseable/pull/23
    • Fix build process for unified binary by @trueleo in https://github.com/parseablehq/parseable/pull/24
    • UI code is now moved to a separate repo by @nitisht in https://github.com/parseablehq/parseable/pull/25
    • Use correct path for frontend resource. by @trueleo in https://github.com/parseablehq/parseable/pull/26
    • Apply clippy suggestions by @trueleo in https://github.com/parseablehq/parseable/pull/27
    • Fix helm chart issues by @nitisht in https://github.com/parseablehq/parseable/pull/30
    • Change stream_name type by @trueleo in https://github.com/parseablehq/parseable/pull/31
    • fix diagram links by @nitisht in https://github.com/parseablehq/parseable/pull/33
    • Add rstest for testing by @trueleo in https://github.com/parseablehq/parseable/pull/32
    • Add serial-test by @trueleo in https://github.com/parseablehq/parseable/pull/34
    • Refactor events by @trueleo in https://github.com/parseablehq/parseable/pull/37
    • Add helm chart initial release by @nitisht in https://github.com/parseablehq/parseable/pull/38
    • Add helm index.yaml and include collector charts here by @nitisht in https://github.com/parseablehq/parseable/pull/39
    • Fix granularity subtraction by @trueleo in https://github.com/parseablehq/parseable/pull/41
    • Add dark mode image link by @nitisht in https://github.com/parseablehq/parseable/pull/44
    • Introduce a new error type for ObjectStorage api by @trueleo in https://github.com/parseablehq/parseable/pull/45
    • Update console release version by @nitisht in https://github.com/parseablehq/parseable/pull/46
    • Fix query validation to ensure we use actual query value by @nitisht in https://github.com/parseablehq/parseable/pull/50
    • Manage sync threads failures by @trueleo in https://github.com/parseablehq/parseable/pull/49
    • feat(alerts): foundational implementation by @de-sh in https://github.com/parseablehq/parseable/pull/28
    • Ignore empty alerts by @trueleo in https://github.com/parseablehq/parseable/pull/59
    • Update Parseable Helm Chart by @AdheipSingh in https://github.com/parseablehq/parseable/pull/57
    • k8s: helm fix mount paths by @AdheipSingh in https://github.com/parseablehq/parseable/pull/60
    • Refactor StorageSync by @trueleo in https://github.com/parseablehq/parseable/pull/55
    • feat: update dependencies by @de-sh in https://github.com/parseablehq/parseable/pull/58
    • Use correct environment variable for validation by @nitisht in https://github.com/parseablehq/parseable/pull/62
    • Update readme by @nitisht in https://github.com/parseablehq/parseable/pull/64
    • Update DO default creds by @nitisht in https://github.com/parseablehq/parseable/pull/66
    • Update logo by @ameen4455 in https://github.com/parseablehq/parseable/pull/67
    • Exact match when deleting stream by @trueleo in https://github.com/parseablehq/parseable/pull/68
    • Migrate to datafusion 11.0 by @nitisht in https://github.com/parseablehq/parseable/pull/65
    • Use common prefixes for logstream listing by @trueleo in https://github.com/parseablehq/parseable/pull/69
    • update readme by @nitisht in https://github.com/parseablehq/parseable/pull/71
    • Fix logstream listing and deletion by @trueleo in https://github.com/parseablehq/parseable/pull/72
    • Use / for listing by @trueleo in https://github.com/parseablehq/parseable/pull/77
    • update .gitignore by @nitisht in https://github.com/parseablehq/parseable/pull/78
    • Move leftover data.parquet to tmp with valid prefix on server startup by @trueleo in https://github.com/parseablehq/parseable/pull/73
    • ObjectStorage: add AuthenticationError type by @123vivekr in https://github.com/parseablehq/parseable/pull/80
    • Collect label and metadata from header by @trueleo in https://github.com/parseablehq/parseable/pull/79
    • Change separator for metadata and tags by @trueleo in https://github.com/parseablehq/parseable/pull/82
    • Update the max number of headers for meta and tags each to 10 by @nitisht in https://github.com/parseablehq/parseable/pull/83
    • Update to latest console release by @nitisht in https://github.com/parseablehq/parseable/pull/84
    • Update to console release 0.0.4 by @nitisht in https://github.com/parseablehq/parseable/pull/85
    • Use existing schema when querying local data. by @trueleo in https://github.com/parseablehq/parseable/pull/86
    • Update readme and upgrade console to 0.0.5 by @nitisht in https://github.com/parseablehq/parseable/pull/88

    New Contributors

    • @trueleo made their first contribution in https://github.com/parseablehq/parseable/pull/11
    • @ameen4455 made their first contribution in https://github.com/parseablehq/parseable/pull/12
    • @de-sh made their first contribution in https://github.com/parseablehq/parseable/pull/28
    • @AdheipSingh made their first contribution in https://github.com/parseablehq/parseable/pull/57
    • @123vivekr made their first contribution in https://github.com/parseablehq/parseable/pull/80

    Full Changelog: https://github.com/parseablehq/parseable/commits/v0.0.1

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(141 bytes)
    parseable_0.0.1_Linux_x86_64(74.37 MB)
    parseable_0.0.1_macOS_x86_64(53.12 MB)
Owner
Parseable, Inc.
Parseable, Inc.
A Modern Real-Time Data Processing & Analytics DBMS with Cloud-Native Architecture, written in Rust

Datafuse Modern Real-Time Data Processing & Analytics DBMS with Cloud-Native Architecture Datafuse is a Real-Time Data Processing & Analytics DBMS wit

Datafuse Labs 5k Jan 4, 2023
Provides multiple-dtype columner storage, known as DataFrame in pandas/R

brassfibre Provides multiple-dtype columner storage, known as DataFrame in pandas/R. Series Single-dtype 1-dimentional vector with label (index). Crea

Sinhrks 21 Nov 28, 2022
Perhaps the fastest and most memory efficient way to pull data from PostgreSQL into pandas and numpy. πŸš€

flaco Perhaps the fastest and most memory efficient way to pull data from PostgreSQL into pandas and numpy. ?? Have a gander at the initial benchmarks

Miles Granger 14 Oct 31, 2022
A fast, powerful, flexible and easy to use open source data analysis and manipulation tool written in Rust

fisher-rs fisher-rs is a Rust library that brings powerful data manipulation and analysis capabilities to Rust developers, inspired by the popular pan

Syed Vilayat Ali Rizvi 5 Aug 31, 2023
A fast, powerful, flexible and easy to use open source data analysis and manipulation tool written in Rust

fisher-rs fisher-rs is a Rust library that brings powerful data manipulation and analysis capabilities to Rust developers, inspired by the popular pan

null 5 Sep 6, 2023
ndarray: an N-dimensional array with array views, multidimensional slicing, and efficient operations

ndarray The ndarray crate provides an n-dimensional container for general elements and for numerics. Please read the API documentation on docs.rs or t

null 2.6k Jan 7, 2023
Dataframe structure and operations in Rust

Utah Utah is a Rust crate backed by ndarray for type-conscious, tabular data manipulation with an expressive, functional interface. Note: This crate w

Suchin 139 Sep 26, 2022
A Rust crate that reads and writes tfrecord files

tfrecord-rust The crate provides the functionality to serialize and deserialize TFRecord data format from TensorFlow. Features Provide both high level

null 22 Nov 3, 2022
Apache Arrow DataFusion and Ballista query engines

DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.

The Apache Software Foundation 2.9k Jan 2, 2023
Fastest and safest Rust implementation of parquet. `unsafe` free. Integration-tested against pyarrow

Parquet2 This is a re-write of the official parquet crate with performance, parallelism and safety in mind. The five main differentiators in compariso

Jorge Leitao 237 Jan 1, 2023
Orkhon: ML Inference Framework and Server Runtime

Orkhon: ML Inference Framework and Server Runtime Latest Release License Build Status Downloads Gitter What is it? Orkhon is Rust framework for Machin

Theo M. Bulut 129 Dec 21, 2022
Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference

Sonos' Neural Network inference engine. This project used to be called tfdeploy, or Tensorflow-deploy-rust. What ? tract is a Neural Network inference

Sonos, Inc. 1.5k Jan 2, 2023
ConnectorX - Fastest library to load data from DB to DataFrames in Rust and Python

ConnectorX enables you to load data from databases into Python in the fastest and most memory efficient way.

SFU Database Group 939 Jan 5, 2023
Provides a way to use enums to describe and execute ordered data pipelines. πŸ¦€πŸΎ

enum_pipline Provides a way to use enums to describe and execute ordered data pipelines. ?? ?? I needed a succinct way to describe 2d pixel map operat

Ben Greenier 0 Oct 29, 2021
AppFlowy is an open-source alternative to Notion. You are in charge of your data and customizations

AppFlowy is an open-source alternative to Notion. You are in charge of your data and customizations. Built with Flutter and Rust.

null 30.7k Jan 7, 2023
New generation decentralized data warehouse and streaming data pipeline

World's first decentralized real-time data warehouse, on your laptop Docs | Demo | Tutorials | Examples | FAQ | Chat Get Started Watch this introducto

kamu 184 Dec 22, 2022
An example repository on how to start building graph applications on streaming data. Just clone and start building πŸ’» πŸ’ͺ

An example repository on how to start building graph applications on streaming data. Just clone and start building ?? ??

Memgraph 40 Dec 20, 2022
Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing

Apache Arrow Powering In-Memory Analytics Apache Arrow is a development platform for in-memory analytics. It contains a set of technologies that enabl

The Apache Software Foundation 10.9k Jan 6, 2023
This library provides a data view for reading and writing data in a byte array.

Docs This library provides a data view for reading and writing data in a byte array. This library requires feature(generic_const_exprs) to be enabled.

null 2 Nov 2, 2022