Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[compute] use auto vectorized compute for some cases #1395

Merged
merged 13 commits into from Aug 11, 2021
Merged

[compute] use auto vectorized compute for some cases #1395

merged 13 commits into from Aug 11, 2021

Conversation

sundy-li
Copy link
Member

@sundy-li sundy-li commented Aug 10, 2021 •
edited

I hereby agree to the terms of the CLA available at: https://datafuse.rs/policies/cla/

Summary

  • Use auto vectorized compute for sum and rem, this can avoid type casting from other types.
  • get instead get_mut
  • Custom Hash functions

Changelog

  • Performance Improvement

Related Issues

Fixes #issue

Test Plan

Unit Tests

Stateless Tests

@databend-bot
Copy link
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@sundy-li
Copy link
Member Author

/run-perf master

@databend-bot
Copy link
Member

run performance on sha f35cf4a reference on master

@sundy-li
Copy link
Member Author

/run-perf master

@databend-bot
Copy link
Member

run performance on sha b033e24 reference on master

@sundy-li
Copy link
Member Author

/run-perf master

@databend-bot
Copy link
Member

run performance on sha 0543fb6 reference on master

@codecov-commenter
Copy link

codecov-commenter commented Aug 10, 2021 •
edited

Codecov Report

Merging #1395 (d836ace) into master (10e793d) will decrease coverage by 0%.
The diff coverage is 36%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #1395   +/-   ##
======================================
- Coverage      72%     72%   -1%     
======================================
  Files         493     493           
  Lines       28526   28591   +65     
======================================
+ Hits        20676   20694   +18     
- Misses       7850    7897   +47
Impacted Files Coverage Δ
common/datavalues/src/data_array_filter.rs 91% <ø> (ø)
...c/pipelines/transforms/transform_group_by_final.rs 76% <ø> (ø)
common/datavalues/src/arrays/arithmetic.rs 26% <4%> (-5%) ⬇️
...pipelines/transforms/transform_group_by_partial.rs 57% <38%> (-21%) ⬇️
common/datavalues/src/arrays/ops/group_hash.rs 53% <50%> (+<1%) ⬆️
common/datavalues/src/arrays/ops/agg.rs 67% <56%> (-3%) ⬇️
common/datavalues/src/arrays/ops/agg_test.rs 90% <100%> (ø)
common/datavalues/src/series/wrap.rs 61% <100%> (+2%) ⬆️
...pelines/transforms/transform_aggregator_partial.rs 87% <0%> (-2%) ⬇️
store/src/meta_service/network.rs 46% <0%> (-2%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 10e793d...d836ace. Read the comment docs.

@sundy-li
Copy link
Member Author

/run-perf master

@databend-bot
Copy link
Member

run performance on sha 766bcaf reference on master

@sundy-li
Copy link
Member Author

/run-perf master

@databend-bot
Copy link
Member

run performance on sha b9da4e9 reference on master

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 10, 2021

please merge latest master patch for perf tests

@sundy-li
Copy link
Member Author

please merge latest master patch for perf tests

Already merged, still failure encount.

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 10, 2021

please merge latest master patch for perf tests

Already merged, still failure encount.

performance dispatch file is outdated, https://github.com/datafuselabs/datafuse/runs/3290701187?check_suite_focus=true

@sundy-li
Copy link
Member Author

/run-perf master

@databend-bot
Copy link
Member

run performance on sha 5b5bad5 reference on master

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 10, 2021

/rerun-perf master

@databend-bot
Copy link
Member

run performance on sha 5b5bad5 reference on master

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 10, 2021

/rerun-perf master

2 similar comments
@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 10, 2021

/rerun-perf master

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 10, 2021

/rerun-perf master

@databend-bot
Copy link
Member

run performance on sha 5b5bad5 reference on master

2 similar comments
@databend-bot
Copy link
Member

run performance on sha 5b5bad5 reference on master

@databend-bot
Copy link
Member

run performance on sha 5b5bad5 reference on master

@sundy-li
Copy link
Member Author

/rerun-perf master

@databend-bot
Copy link
Member

run performance on sha 8b5b1ee reference on master

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 11, 2021

/run-perf master

@databend-bot
Copy link
Member

run performance on sha 8b5b1ee reference on master

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Aug 11, 2021

/run-perf master

@databend-bot
Copy link
Member

run performance on sha 8b5b1ee reference on master

@sundy-li sundy-li marked this pull request as ready for review August 11, 2021 03:52
zhang2014
zhang2014 approved these changes Aug 11, 2021
View reviewed changes
Copy link
Member

@zhang2014 zhang2014 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect

@databend-bot
Copy link
Member

CI Passed
Reviewer Approved
Let's Merge

@databend-bot databend-bot merged commit c02f15c into datafuselabs:master Aug 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhang2014 zhang2014 zhang2014 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@sundy-li @databend-bot @codecov-commenter @ZhiHanZ @zhang2014