Error

./tinysearch public/index.json
Error: NotEnoughSpace

Index file

 ls -lha public/index.json
-rw-r--r-- 1 q q 6,7M Jun 11 22:46 public/index.json

strace results

strace ./tinysearch public/index.json -p /tmp/

...

getrandom("", 0, GRND_NONBLOCK)         = 0
getrandom("\x1d\xee\x9e\xa5\x0c\x2d\xf4\x99\x9c\x9a\xe5\xb8\xc4\x52\x05\xb7\x4c\x3c\x2a\x79\xe4\xee\x5f\x24\xaf\x1e\xcf\x62\xb8\x49\x35\x97", 32, GRND_NONBLOCK) = 32
write(2, "Error: ", 7Error: )                  = 7
write(2, "NotEnoughSpace", 14NotEnoughSpace)          = 14
write(2, "\n", 1
)                       = 1
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=8192}, NULL) = 0
munmap(0x7f7fffdd5000, 12288)           = 0
exit_group(1)                           = ?
+++ exited with 1 +++

How difficult would it be to also return the page description in the results?

I have been implementing tinysearch into the abridge zola theme.

here is a demo using tinysearch: https://jieiku.github.io/abridge-tinysearch/

here is the normal demo which is using elasticlunr (zola's default): https://abridge.netlify.app/

abridge tinysearch branch: https://github.com/Jieiku/abridge/tree/tinysearch

Here is how elasticlunr looks:

I have modified the tinysearch js example to more tightly integrate with abridge, so it looks VERY similar, but I don't have a description to go along with the title:

Might be a fix for https://github.com/tinysearch/tinysearch/issues/152, https://github.com/tinysearch/tinysearch/issues/151, https://github.com/tinysearch/tinysearch/issues/147. Would be delighted if someone could test this.

I'm evaluating Tinysearch for my hugo blog. I built the json by tweaking a little bit the example given and compiled tinysearch from git.

I installed it here for now so you can try it. The index is here if you want to find something in here :-)

• It finds results from words which doesn't exist at all ( try "xugn.ui").
• It finds pages without the term (try "morse").
• it can't find pages containing the term (try "midsommar")

Is the index not well built or is a bug or else ?

Trying to run tinysearch on Windows 11 (with a Hugo blog):

> tinysearch.exe .\public\index.json
Unpacking tinysearch WASM engine into temporary directory "C:\\Users\\stop_\\AppData\\Local\\Temp\\.tmpZx9gHh"
Starting unpack
Name: \\?\C:\Users\stop_\(redacted)\blog\.git
Name: \\?\C:\Users\stop_\(redacted)\blog\.gitmodules
Name: \\?\C:\Users\stop_\(redacted)\blog\.hugo_build.lock
Name: \\?\C:\Users\stop_\(redacted)\blog\archetypes
Name: \\?\C:\Users\stop_\(redacted)\blog\config.toml
Name: \\?\C:\Users\stop_\(redacted)\blog\content
Name: \\?\C:\Users\stop_\(redacted)\blog\data
Name: \\?\C:\Users\stop_\(redacted)\blog\go.mod
Name: \\?\C:\Users\stop_\(redacted)\blog\go.sum
Name: \\?\C:\Users\stop_\(redacted)\blog\layouts
Name: \\?\C:\Users\stop_\(redacted)\blog\public
Name: \\?\C:\Users\stop_\(redacted)\blog\resources
Name: \\?\C:\Users\stop_\(redacted)\blog\static
Name: \\?\C:\Users\stop_\(redacted)\blog\storage
Name: \\?\C:\Users\stop_\(redacted)\blog\themes
Error: Engine directory could not be created at C:\Users\stop_\AppData\Local\Temp\.tmpZx9gHh\engine

This is a quite incomplete error message in my opinion...

Hello,

I am trying to test the docker but when running docker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly .

at step 19 from 28, the build stops with the error =>

Step 19/28 : RUN wasm-pack --version
 ---> Running in d0787575a530
The command '/bin/sh -c wasm-pack --version' returned a non-zero code: 139

I tried with others docker images like docker build --build-arg WASM_BRANCH=master --build-arg TINY_MAGIC=64 -t tinysearch/cli . or other solutions from this thread => https://github.com/tinysearch/tinysearch/issues/111

But it always lead to this error.

Does anyone know why this happened?.

Thanks for the help.

Just a proof of concept for now - should solve the compile times issue since the wasm can be compiled once and then filters generated separately. To test, build the project as normal then overwrite the .wasm files by running (in the engine dir):

tinysearch/engine $ wasm-pack build --target web --release --out-dir ../

Next steps:

1. Separate out the wasm from the filter generation, publish to npm
2. Build a cli for generating the filters

Let me know if you think it's worth continuing

Cheers,

P

At the moment, it takes ages to generate the Wasm binary. That's because we download an entire crate and compile it whenever we run tinysearch corpus.json. Maybe anyone has an idea on how to improve that?

When I was running tinysearch locally and ran cargo run fixtures/index.json, it was wiping my local repository and replacing it with the wasm output.

The above PR changes the default directory when no output directory is specified to a "wasm_output" dir created in the local directory.

I'm curious about how the current tinysearch cargo build would work in another web project and if it would delete all the files and replace them with the wasm output.

Also please excuse if there's anything wrong with my code, I'm new to Rust.

Hi! Can you please add the tag v0.7.0 to point to https://github.com/tinysearch/tinysearch/commit/5d98c355e4f6a0e8c7f67e5764cd0322883ecdef ?

This would enable bumping the Homebrew formula to the latest version! Thank you!

For some use-cases, Xor filters are supposed to be faster and smaller than Bloom and Cuckoo filters. Recently, support for strings was added to the xorfilter Rust crate: https://github.com/bnclabs/xorfilter/blob/master/tests/xorfilter.rs#L56 We should test that to see if they make our wasm binary smaller.

If someone wants to give this a shot, please write a short comment here. I'd be happy to provide mentoring if needed.

For Zola sites, the tinysearch json index gets included in the sitemap.xml file.

https://endler.dev/sitemap.xml

I found a report that suggests not rendering the section will resolve the issue, but thats not the case for me: https://github.com/getzola/zola/issues/604

There is currently a pull request to allow more formats, maybe then we could output/use json instead of html? https://github.com/getzola/zola/pull/1998

After struggling mightily with tinysearch installed via Homebrew and direct downloads, this one was installed via cargo install tinysearch, cargo install wasm-pack

Help please, I am still stuck :(

Why is tinysearch's invocation of wasm-pack looking for a Cargo.toml file in the temp directory it created?

tinysearch --optimize --path static public/data_tinysearch/index.html
Unpacking tinysearch WASM engine into temporary directory "/var/folders/nx/7ys_xr014yg41_mw1zn_wvb00000gn/T/.tmpMxhsOO"
Starting unpack
Copying index into crate
Compiling WASM module using wasm-pack
Error: crate directory is missing a `Cargo.toml` file; is `/var/folders/nx/7ys_xr014yg41_mw1zn_wvb00000gn/T/.tmpMxhsOO/engine` the wrong directory?
Error: failed to execute "wasm-pack" "build" "/var/folders/nx/7ys_xr014yg41_mw1zn_wvb00000gn/T/.tmpMxhsOO/engine" "--target" "web" "--release" "--out-dir" "/Users/name/blog/static"
status: exit status: 1

Does tinysearch support CJK languages? (Chinese, Japanese, and Korean)

Also what about stemmers and stopwords?

I am interested in using tinysearch for Zola. I proposed it here: https://github.com/getzola/zola/issues/1849

It was mentioned that there may not be specific stemmers/stopword lists for languages other than English?

I was reading over this https://endler.dev/2019/tinysearch/ and I seen removing stopwords mentioned, but it was not clear if this was something you manually did or something that is supported by tinysearch when it builds the index.

The tinysearch fixtures/index.json command in the "Usage" section of the readme isn't working. I think this is because the Cargo build for tinysearch hasn't been changed since the fix in this PR: https://github.com/tinysearch/tinysearch/pull/154.

When I ran the local project using cargo run fixtures/index.json, the generation worked as expected.

When trying to build nightly with docker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly . I'm getting the following error:

thread 'main' panicked at 'crate directory should exist', src/readme.rs:11:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Well, this is embarrassing.

wasm-pack had a problem and crashed. To help us diagnose the problem you can send us a crash report.


We have generated a report file at "/tmp/report-2ec64b30-c702-4581-a790-5d71fb2f66fe.toml". Submit an issue or email with the subject of "wasm-pack Crash Report" and include the report as an attachment.

- Authors: Ashley Williams <[email protected]>, Jesper Håkansson <[email protected]>

We take privacy seriously, and do not perform any automated error collection. In order to improve the software, we rely on people to submit reports.

Thank you kindly!
Error: failed to execute "wasm-pack" "build" "/tmp/.tmpzklwBA/engine" "--target" "web" "--release" "--out-dir" "/tmp"
status: exit status: 101

In spite of the message no report file was generated at tmp/report-2ec64b30-c702-4581-a790-5d71fb2f66fe.toml

I guess I have pushed capabilities of tinysearch to the limits. The site I'm indexing is by no means tiny. It's over 1700 pages albeit with very little text. The compiled wasm binary is 343k . Amazingly it still works very fast, though in case of some queries I'm getting a lot of false positives, like over 50%. I was wondering if it might be possible to make it more precise at the cost of a larger binary. I could live up with a ca 1MB binary as this site definitely does not have to load under 3s. I was using the docker image to compile the binary.

docker run -v $PWD:/tmp tinysearch/cli index.json