New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[diemdb] add API get_transaction_by_hash #9206
Conversation
.get_transaction_by_hash(txn_to_commit.transaction().hash(), ledger_version, true) | ||
.unwrap() | ||
.unwrap(); | ||
txn_with_proof |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably assert_eq!(txn_with_proof.transaction().hash(), txn_to_commit.transaction().hash())
as well, which is the user intended way to use it? @xli
) -> Result<Option<TransactionWithProof>> { | ||
let version = match self | ||
.transaction_store | ||
.get_transaction_version_by_hash(&hash, ledger_version)? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use Option::map()
?
storage/storage-interface/src/lib.rs
Outdated
@@ -190,6 +190,16 @@ pub trait DbReader: Send + Sync { | |||
fetch_events: bool, | |||
) -> Result<TransactionListWithProof>; | |||
|
|||
/// See [`DiemDB::get_transaction_by_hash`]. | |||
/// | |||
/// [`DiemDB::get_transactions`]: ../diemdb/struct.DiemDB.html#method.get_transaction_by_hash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be nice to note it somewhere that we can't prove absence.
storage/storage-interface/src/lib.rs
Outdated
hash: HashValue, | ||
ledger_version: Version, | ||
fetch_events: bool, | ||
) -> Result<Option<TransactionWithProof>>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Returning None might not give enough detail to the user.
- Is there is a range of ledger versions can't we prove absence? - Does this node only have the latest version?
- Should we let the user know the version range that was searched if it can't be found?
- Should we also let the user search a particular version range?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- l think I can remove the ledger_version as it is best-effort. We cannot prove absence for any version.
- Even we let user know, it doesn't help because there is no proof and the reason can be multifold.
- If we cannot prove, why do we need version range?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- return only "if txn.version < ledger_version" would be the most we want to do :P or remove the parameter.
- I don't think the complexity is worth it. We just put in disclaimers to say we don't guarantee absence and can't prove it. In reality everything after the API is supported will be searched and returned, but again we can't prove it.
- no
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume a reasonable use case is something like:
I want to find my txn so do you have it? Server says no.
Do I look elsewhere? If the server searched versions 0 current, then I don't need to (if I trust this server).
Otherwise if the server only had current_minus_one to current, I should absolutely ask another server who can tell me 0 to currrent_minus_one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The use case is I just submitted a transaction and I want to see if it's included yet, if I didn't misunderstand.
We always have the proper APIs (get_by_addr_and_seq(), get_by_ver()) to get historical transactions.
I'd rather always guarantee (0, ledger_version) is searched by backfilling the data, than complicating the API for ever.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we return OK(None) is NOT_FOUND, other query errors are Err(_). But the reasons of NOT_FOUND can be various. In our case, the index may somehow get lost or the ledger version is too new.
the ledger version there is to keep a consistent view of api, to make the client know not_found returned by querying the server's local range but not beyond server's local version range.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think index data lost probably can be ignored if we don't have a way to distinguish it from not exist.
We should assume index (0, ledger_version) will be eventually fulfilled. (should we have an admin api to rebuild index in background for node?)
Is there a plan to prune this index for old txn data? I assumed not, once we build the index, we will keep it, right? if that's the case, I think return Ok(None) is fine to me for the case not found.
Service layer will return current latest ledger version in response body when txn is not found.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For admin api, that's a possible way to go. But maybe not hi-pri for now.
I think the pruning will be consistent with the txn, if txn got pruned, there is no sense to keep the index. If this case, how do you run the admin api?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Synced offline with our "customer" @xli, and the team agreed on moving forward with the simple interface.
@aching My thinking is, if pruning is in place, with this simple interface, we have two options: 1. maintain a strong guarantee that the index searched covers all transactions that's on the server, in that case, the JsonAPI layer will have way to know the range. or 2. as we claim today, we don't make any guarantee that the index is complete (in reality today though, our index covers all transactions created after the new version of software is put in, and if we do a backfill, that equals the full range of existing transactions.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All sounds good to me, one ask for the case that no guarantee the index is complete: could we create a document for how to make the index complete (when we have the way to backfill the index), including configurations required for keep index after built?
@bors-libra land |
💔 Test Failed - ci-test |
/land |
💔 Test Failed - ci-test |
f0aff83
to
672e11e
Compare
@bors-libra land |
Forge Test Result
Repro cmd:
🎉 Land-blocking forge test passed! 👌 |
Cluster Test Result
Repro cmd:
🎉 Land-blocking cluster test passed! 👌 |
672e11e
to
82a75c3
Compare
Motivation
Store the hash->version to db and add the API to get the transaction by hash.
Have you read the Contributing Guidelines on pull requests?
Y
Test Plan
current coverage.