Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object does not support containing Chinese characters #260

Closed
springe2004 opened this issue Jul 28, 2021 · 3 comments
Closed

Object does not support containing Chinese characters #260

springe2004 opened this issue Jul 28, 2021 · 3 comments

Comments

@springe2004
Copy link

(DEBUG) - got channel message: PUSH ChatUser 7dcb2a5c-89d6-4260-bc3e-bce73dbb0c6b {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"} "{"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}"
(DEBUG) - will dispatch search command: PUSH
(DEBUG) - parsed text parts (still needs post-processing): "{"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中 文名"}"
(DEBUG) - parsed text parts (post-processed): {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
(DEBUG) - dispatching ingest push in collection: ChatUser, bucket: 7dcb2a5c-89d6-4260-bc3e-bce73dbb0c6b and object: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
(DEBUG) - ingest push has text: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
(DEBUG) - will push for text: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"} with hinted locale:
(DEBUG) - detecting locale from lexer text: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
(DEBUG) - will detect locale for lexer safe text: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
(DEBUG) - lexer text is equal or longer than 60 characters, using the fast method
(DEBUG) - guessing locale from stopwords for script: Latin and text: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
(DEBUG) - [fast lexer] trying to detect locale from fallback ngram instead
(DEBUG) - wrote response with values: ERR (query_error)

@valeriansaliou
Copy link
Owner

To be clear are you pushing CJK characters in the terms section or as an object_id? (object_id has to be alphanumeric)

@springe2004
Copy link
Author

collection:ChatUser
bucket:default
object: {"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}
text:{"Id":"6489a4aa-d238-e468-9081-2afdf2a32156","NickName":"中文名"}

the object is a json

@valeriansaliou
Copy link
Owner

You should not inject bare JSON into Sonic. Sonic does not understand JSON. If you want to index multiple key values for a given object, what I recommend is to unpack the JSON values and concatenate them all together, space-separated.

Eg.: {"first_name": "Valerian", "last_name": "Saliou"} would become: Valerian Saliou

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants