-
Notifications
You must be signed in to change notification settings - Fork 151
Public API RFC #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
c46004d
91e20f5
c993330
403a9d5
ec9e8dd
01b0362
87ad84c
48a11e1
d3043d5
a76504f
46a8d05
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| [package] | ||
| name = "tikv-client" | ||
| version = "0.0.0" | ||
| keywords = ["TiKV", "KV", "distributed-systems"] | ||
| license = "Apache-2.0" | ||
| authors = ["The TiKV Project Developers"] | ||
| repository = "https://github.com/tikv/client-rust" | ||
| description = "The rust language implementation of TiKV client." | ||
|
|
||
| [lib] | ||
| name = "tikv_client" | ||
|
|
||
| [dependencies] | ||
| futures = "0.1" | ||
| serde = "1.0" | ||
| serde_derive = "1.0" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| extern crate futures; | ||
| extern crate serde; | ||
| #[macro_use] | ||
| extern crate serde_derive; | ||
|
|
||
| use std::io::Error; | ||
| use std::path::PathBuf; | ||
|
|
||
| use futures::Future; | ||
|
|
||
| pub mod raw; | ||
| pub mod transaction; | ||
|
|
||
| #[derive(Default, Clone, Eq, PartialEq, Ord, PartialOrd, Hash, Debug)] | ||
| pub struct Key(Vec<u8>); | ||
| #[derive(Default, Clone, Eq, PartialEq, Hash, Debug)] | ||
| pub struct Value(Vec<u8>); | ||
| #[derive(Default, Clone, Eq, PartialEq, Debug)] | ||
| pub struct KvPair(Key, Value); | ||
| #[derive(Default, Clone, Eq, PartialEq, Debug)] | ||
| pub struct KeyRange(Key, Key); | ||
|
|
||
| pub type KvFuture<T> = Box<Future<Item = T, Error = Error> + Send>; | ||
|
|
||
| impl Into<Key> for Vec<u8> { | ||
| fn into(self) -> Key { | ||
| Key(self) | ||
| } | ||
| } | ||
|
|
||
| impl Into<Value> for Vec<u8> { | ||
| fn into(self) -> Value { | ||
| Value(self) | ||
| } | ||
| } | ||
|
|
||
| impl Into<KvPair> for (Key, Value) { | ||
| fn into(self) -> KvPair { | ||
| KvPair(self.0, self.1) | ||
| } | ||
| } | ||
|
|
||
| impl Into<KeyRange> for (Key, Key) { | ||
| fn into(self) -> KeyRange { | ||
| KeyRange(self.0, self.1) | ||
| } | ||
| } | ||
|
|
||
| #[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq)] | ||
| #[serde(default)] | ||
| #[serde(rename_all = "kebab-case")] | ||
| pub struct Config { | ||
| pub pd_endpoints: Vec<String>, | ||
| pub ca_path: Option<PathBuf>, | ||
| pub cert_path: Option<PathBuf>, | ||
| pub key_path: Option<PathBuf>, | ||
| } | ||
|
|
||
| impl Config { | ||
| pub fn new<E>(pd_endpoints: E) -> Self | ||
| where | ||
| E: IntoIterator<Item = String>, | ||
| { | ||
| Config { | ||
| pd_endpoints: pd_endpoints.into_iter().collect(), | ||
| ca_path: None, | ||
| cert_path: None, | ||
| key_path: None, | ||
| } | ||
| } | ||
|
|
||
| pub fn with_security<E>( | ||
| pd_endpoints: E, | ||
| ca_path: PathBuf, | ||
| cert_path: PathBuf, | ||
| key_path: PathBuf, | ||
| ) -> Self | ||
| where | ||
| E: IntoIterator<Item = String>, | ||
| { | ||
| Config { | ||
| pd_endpoints: pd_endpoints.into_iter().collect(), | ||
| ca_path: Some(ca_path), | ||
| cert_path: Some(cert_path), | ||
| key_path: Some(key_path), | ||
| } | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| use {Config, Key, KeyRange, KvFuture, KvPair, Value}; | ||
|
|
||
| pub trait Client { | ||
| fn new(_config: &Config) -> KvFuture<Self> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn get<K, C>(&self, key: K, cf: C) -> KvFuture<Value> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe we can borrow the key here? This would possibly save the caller a
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The underlying grpc request is going to move this key anyway. And I found there is a suggestion in API Guideline saying if an argument is going to be moved, it's better to make it clear on function signature instead of making a clone/copy inside the implementation. |
||
| where | ||
| K: Into<Key>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn batch_get<I, K, C>(&self, keys: I, cf: C) -> KvFuture<Vec<KvPair>> | ||
| where | ||
| I: IntoIterator<Item = K>, | ||
| K: Into<Key>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn put<P, C>(&self, pair: P, cf: C) -> KvFuture<()> | ||
| where | ||
| P: Into<KvPair>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn batch_put<I, P, C>(&self, pairs: I, cf: C) -> KvFuture<()> | ||
| where | ||
| I: IntoIterator<Item = P>, | ||
| P: Into<KvPair>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn delete<K, C>(&self, key: K, cf: C) -> KvFuture<()> | ||
| where | ||
| K: Into<Key>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn batch_delete<I, K, C>(&self, keys: I, cf: C) -> KvFuture<()> | ||
| where | ||
| I: IntoIterator<Item = K>, | ||
| K: Into<Key>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn scan<R, C>(&self, range: R, limit: u32, key_only: bool, cf: C) -> KvFuture<Vec<KvPair>> | ||
| where | ||
| R: Into<KeyRange>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn batch_scan<I, R, C>( | ||
| &self, | ||
| ranges: I, | ||
| each_limit: u32, | ||
| key_only: bool, | ||
| cf: C, | ||
| ) -> KvFuture<Vec<KvPair>> | ||
| where | ||
| I: IntoIterator<Item = R>, | ||
| R: Into<KeyRange>, | ||
| C: Into<Option<String>>; | ||
|
|
||
| fn delete_range<R, C>(&self, range: R, cf: C) -> KvFuture<()> | ||
| where | ||
| R: Into<KeyRange>, | ||
| C: Into<Option<String>>; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,229 @@ | ||
| use std::io::Error; | ||
|
|
||
| use futures::{Poll, Stream}; | ||
|
|
||
| use {Config, Key, KvFuture, KvPair, Value}; | ||
|
|
||
| #[derive(Copy, Clone)] | ||
| pub struct Timestamp(u64); | ||
|
|
||
| impl Into<Timestamp> for u64 { | ||
| fn into(self) -> Timestamp { | ||
| Timestamp(self) | ||
| } | ||
| } | ||
|
|
||
| impl Timestamp { | ||
| pub fn timestamp(self) -> u64 { | ||
| self.0 | ||
| } | ||
|
|
||
| pub fn physical(self) -> i64 { | ||
| (self.0 >> 16) as i64 | ||
| } | ||
|
|
||
| pub fn logical(self) -> i64 { | ||
| (self.0 & 0xFFFF as u64) as i64 | ||
| } | ||
| } | ||
|
|
||
| pub struct Scanner; | ||
|
|
||
| impl Stream for Scanner { | ||
| type Item = KvPair; | ||
| type Error = Error; | ||
|
|
||
| fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> { | ||
| unimplemented!() | ||
| } | ||
| } | ||
|
|
||
| pub trait Retriever { | ||
| fn get<K>(&self, key: K) -> KvFuture<Value> | ||
| where | ||
| K: Into<Key>; | ||
|
|
||
| fn batch_get<I, K>(&self, keys: I) -> KvFuture<Vec<KvPair>> | ||
| where | ||
| I: IntoIterator<Item = K>, | ||
| K: Into<Key>; | ||
|
|
||
| fn seek<K>(&self, key: K) -> KvFuture<Scanner> | ||
| where | ||
| K: Into<Key>; | ||
|
|
||
| fn seek_reverse<K>(&self, key: K) -> KvFuture<Scanner> | ||
| where | ||
| K: Into<Key>; | ||
| } | ||
|
|
||
| pub trait Mutator { | ||
| fn set<P>(&mut self, pair: P) -> KvFuture<()> | ||
| where | ||
| P: Into<KvPair>; | ||
|
|
||
| fn delete<K>(&mut self, key: K) -> KvFuture<()> | ||
| where | ||
| K: Into<Key>; | ||
| } | ||
|
|
||
| pub struct Transaction; | ||
|
|
||
| impl Transaction { | ||
| pub fn commit(&mut self) -> KvFuture<()> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| pub fn rollback(&mut self) -> KvFuture<()> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| pub fn lock_keys<I, K>(&mut self, keys: I) -> KvFuture<()> | ||
| where | ||
| I: IntoIterator<Item = I>, | ||
| K: Into<Key>, | ||
| { | ||
| drop(keys); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| pub fn is_readonly(&self) -> bool { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| pub fn start_ts(&self) -> Timestamp { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| pub fn snapshot(&self) -> KvFuture<Snapshot> { | ||
| unimplemented!() | ||
| } | ||
| } | ||
|
|
||
| impl Retriever for Transaction { | ||
| fn get<K>(&self, key: K) -> KvFuture<Value> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn batch_get<I, K>(&self, keys: I) -> KvFuture<Vec<KvPair>> | ||
| where | ||
| I: IntoIterator<Item = K>, | ||
| K: Into<Key>, | ||
| { | ||
| drop(keys); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn seek<K>(&self, key: K) -> KvFuture<Scanner> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn seek_reverse<K>(&self, key: K) -> KvFuture<Scanner> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
| } | ||
|
|
||
| impl Mutator for Transaction { | ||
| fn set<P>(&mut self, pair: P) -> KvFuture<()> | ||
| where | ||
| P: Into<KvPair>, | ||
| { | ||
| drop(pair); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn delete<K>(&mut self, key: K) -> KvFuture<()> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
| } | ||
|
|
||
| pub struct Snapshot; | ||
|
|
||
| impl Retriever for Snapshot { | ||
| fn get<K>(&self, key: K) -> KvFuture<Value> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn batch_get<I, K>(&self, keys: I) -> KvFuture<Vec<KvPair>> | ||
| where | ||
| I: IntoIterator<Item = K>, | ||
| K: Into<Key>, | ||
| { | ||
| drop(keys); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn seek<K>(&self, key: K) -> KvFuture<Scanner> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn seek_reverse<K>(&self, key: K) -> KvFuture<Scanner> | ||
| where | ||
| K: Into<Key>, | ||
| { | ||
| drop(key); | ||
| unimplemented!() | ||
| } | ||
| } | ||
|
|
||
| pub struct Oracle; | ||
|
|
||
| impl Oracle { | ||
| pub fn timestamp(&self) -> KvFuture<Timestamp> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| pub fn is_expired(&self, _lock_timestamp: Timestamp, _ttl: Timestamp) -> KvFuture<bool> { | ||
| unimplemented!() | ||
| } | ||
| } | ||
|
|
||
| pub trait Client { | ||
| fn new(_config: &Config) -> KvFuture<Self> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn begin(&self) -> KvFuture<Transaction> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn begin_with_timestamp(&self, _timestamp: Timestamp) -> KvFuture<Transaction> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn snapshot(&self) -> KvFuture<Snapshot> { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn current_timestamp(&self) -> Timestamp { | ||
| unimplemented!() | ||
| } | ||
|
|
||
| fn oracle(&self) -> Oracle { | ||
| unimplemented!() | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mostly, we name our Go library as ticlient, maybe we can use the same name here. :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/cc @c4pt0r
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had a little concern what if there are other storage system built on top of TiKV and we want to implement a Rust client for it as well?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although it's probably too early to say that. But I really pictured a day in my mind that we could build a distributed file system or maybe timeseries database on top of TiKV. And for the nature of it's protocol complexity, it's likely a dedicated client is preferred.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't believe that after more than a decade, HDFS probably wins in ecosystem but still loses in scalability due to HA NameNode architecture. Even Google's initial GFS has something better than HDFS. Although there are federation support but it sounds better to have a clustered NameNode natively which can be built on top of TiKV.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@siddontang I can only see
TiClientused twice in TiDB's code: https://github.com/pingcap/tidb/search?q=TiClient&unscoped_q=TiClient.Also in the name of one test file https://github.com/pingcap/tidb/blob/cb03f2bec151033a38dfd17715a42cf3728b0541/store/tikv/ticlient_test.go
I think using a different name than
tikv_clientrisks the user not understanding what this is for.ticlientis ambiguous and could meantidbortisparkor something else.