Skip to content

Commit 52a69e1

Browse files
robotraptarobotrapta
committed
Updating README and Developer guide.
1 parent d9a0084 commit 52a69e1

File tree

4 files changed

+196
-277
lines changed

4 files changed

+196
-277
lines changed

‎DEVELOPING.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Python SDK Developer Guide
2+
3+
The raw API is generated using an OpenAPI spec. The SDK adds commonly used functionality
4+
like polling for blocking submits and configuration of tokens and endpoints.
5+
6+
## Local Development
7+
8+
The auto-generated SDK code is in the `generated/` directory. To
9+
re-generate the client code, you'll need to install
10+
[openapi-generator](https://openapi-generator.tech/docs/installation#homebrew)
11+
(I recommend homebrew if you're on a mac). Then you can run it with:
12+
13+
```Bash
14+
$ make generate
15+
```
16+
17+
## Testing
18+
Most tests need an API endpoint to run. This can be the public API endpoint `https://api.groundlight.ai`,
19+
or a local endpoint with an edge client or development environment.
20+
21+
### Getting the tests to use your current code.
22+
23+
(This needs to be updated.)
24+
25+
You kinda want to do a `pip install -e .` equivalent but I don't know how to do that with poetry. The ugly version is this...
26+
27+
Find the directory where `groundlight` is installed:
28+
29+
```
30+
$ python
31+
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
32+
[GCC 7.3.0] :: Anaconda, Inc. on linux
33+
Type "help", "copyright", "credits" or "license" for more information.
34+
>>> import groundlight
35+
>>> groundlight
36+
<module 'groundlight' from '/home/ubuntu/anaconda3/lib/python3.7/site-packages/groundlight/__init__.py'>
37+
```
38+
39+
Then blow this away and set up a symlink from that directory to your source.
40+
41+
```
42+
cd /home/ubuntu/anaconda3/lib/python3.7/site-packages/
43+
rm -rf groundlight
44+
ln -s ~/dev/python-sdk/src/groundlight groundlight
45+
```

‎README.md‎

Lines changed: 150 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -1,126 +1,182 @@
1-
# Python SDK Internal README
2-
# (See [UserGuide](UserGuide.md) for the public README)
1+
# Groundlight Python SDK
32

4-
This package builds the SDK which is an easier-to-use wrapper around the public API.
5-
The raw API is generated using an OpenAPI spec. But then we add functionality here in the SDK
6-
for things like blocking submit and configuration of tokens and endpoints.
3+
Groundlight makes it simple to understand images. You can easily create computer vision detectors just by describing what you want to know using natural language. Groundlight uses a combination of advanced AI and real-time human monitors to automatically turn your images and queries into a customized machine learning (ML) model for your application.
74

8-
The SDK is published through github actions to pypi at [https://pypi.org/project/groundlight/](https://pypi.org/project/groundlight/).
5+
## Computer vision made simple
96

10-
### Usage
7+
How to build a working computer vision system in just 5 lines of python code:
8+
9+
```Python
10+
from groundlight import Groundlight
11+
gl = Groundlight()
12+
d = gl.get_or_create_detector(name="door", query="Is the door open?") # define with natural language
13+
image_query = gl.submit_image_query(detector=d, image=jpeg_img) # send in an image
14+
print(f"The answer is {image_query.result}") # get the result
15+
```
16+
17+
**How does it work?** Your images are first analyzed by machine learning (ML) models which are automatically trained on your data. If those models have high enough confidence, that's your answer. But if the models are unsure, then the images are progressively escalated to more resource-intensive analysis methods up to real-time human review. So what you get is a computer vision system that starts working right away without even needing to first gather and label a dataset. At first it will operate with high latency, because people need to review the image queries. But over time, the ML systems will learn and improve so queries come back faster with higher confidence.
18+
19+
*Note: The SDK is currently in "beta" phase. Interfaces are subject to change in future versions.*
1120

12-
For instructions on using the SDK see the public [User Guide](UserGuide.md).
1321

14-
For more details, see the [Groundlight](src/groundlight/client.py)
15-
class. This SDK closely follows the methods in our [API
16-
Docs](https://app.groundlight.ai/reef/admin/public-api-docs/).
22+
## Managing confidence levels and latency
1723

18-
## Development
24+
Groundlight gives you a simple way to control the trade-off of latency against accuracy. The longer you can wait for an answer to your image query, the better accuracy you can get. In particular, if the ML models are unsure of the best response, they will escalate the image query to more intensive analysis with more complex models and real-time human monitors as needed. Your code can easily wait for this delayed response. Either way, these new results are automatically trained into your models so your next queries will get better results faster.
1925

20-
The auto-generated SDK code is in the `generated/` directory. To
21-
re-generate the client code, you'll need to install
22-
[openapi-generator](https://openapi-generator.tech/docs/installation#homebrew)
23-
(I recommend homebrew if you're on a mac). Then you can run it with:
26+
The desired confidence level is set as the escalation threshold on your detector. This determines what is the minimum confidence score for the ML system to provide before the image query is escalated.
2427

25-
```Bash
26-
$ make generate
28+
For example, say you want to set your desired confidence level to 0.95, but that you're willing to wait up to 60 seconds to get a confident response.
29+
30+
```Python
31+
d = gl.get_or_create_detector(name="trash", query="Is the trash can full?", confidence=0.95)
32+
image_query = gl.submit_image_query(detector=d, image=jpeg_img, wait=60)
33+
# This will wait until either 60 seconds have passed or the confidence reaches 0.95
34+
print(f"The answer is {image_query.result}")
2735
```
2836

29-
## Testing
30-
Most tests need an API endpoint to run.
37+
Or if you want to run as fast as possible, set `wait=0`. This way you will only get the ML results, without waiting for escalation. Image queries which are below the desired confidence level still be escalated for further analysis, and the results are incorporated as training data to improve your ML model, but your code will not wait for that to happen.
38+
39+
```Python
40+
image_query = gl.submit_image_query(detector=d, image=jpeg_img, wait=0)
41+
```
3142

32-
### Getting the tests to use your current code.
43+
If the returned result was generated from an ML model, you can see the confidence score returned for the image query:
44+
45+
```Python
46+
print(f"The confidence is {image_query.result.confidence}")
47+
```
48+
49+
## Getting Started
50+
51+
1. Install the `groundlight` SDK. Requires python version 3.7 or higher. See [prerequisites](#Prerequisites).
52+
53+
```Bash
54+
$ pip3 install groundlight
55+
```
3356

34-
You kinda want to do a `pip install -e .` equivalent but I don't know how to do that with poetry. The ugly version is this...
57+
1. To access the API, you need an API token. You can create one on the
58+
[groundlight web app](https://app.groundlight.ai/reef/my-account/api-tokens).
3559

36-
Find the directory where `groundlight` is installed:
60+
The API token should be stored securely. You can use it directly in your code to initialize the SDK like:
3761

62+
```python
63+
gl = Groundlight(api_token="<YOUR_API_TOKEN>")
3864
```
39-
$ python
40-
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
41-
[GCC 7.3.0] :: Anaconda, Inc. on linux
42-
Type "help", "copyright", "credits" or "license" for more information.
43-
>>> import groundlight
44-
>>> groundlight
45-
<module 'groundlight' from '/home/leo/anaconda3/lib/python3.7/site-packages/groundlight/__init__.py'>
65+
66+
which is an easy way to get started, but is NOT a best practice. Please do not commit your API Token to version control! Instead we recommend setting the `GROUNDLIGHT_API_TOKEN` environment variable outside your code so that the SDK can find it automatically.
67+
68+
```bash
69+
$ export GROUNDLIGHT_API_TOKEN=api_2GdXMflhJi6L_example
70+
$ python3 glapp.py
4671
```
4772

48-
Then blow this away and set up a symlink from that directory to your source.
73+
74+
75+
## Prerequisites
76+
77+
### Using Groundlight SDK on Ubuntu 18.04
78+
79+
Ubuntu 18.04 still uses python 3.6 by default, which is end-of-life. We recommend setting up python 3.8 as follows:
4980

5081
```
51-
cd /home/leo/anaconda3/lib/python3.7/site-packages/
52-
rm -rf groundlight
53-
ln -s ~/ptdev/groundlight-python-sdk/src/groundlight groundlight
82+
# Prepare Ubuntu to install things
83+
sudo apt-get update
84+
# Install the basics
85+
sudo apt-get install -y python3.8 python3.8-distutils curl
86+
# Configure `python3` to run python3.8 by default
87+
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 10
88+
# Download and install pip3.8
89+
curl https://bootstrap.pypa.io/get-pip.py > /tmp/get-pip.py
90+
sudo python3.8 /tmp/get-pip.py
91+
# Configure `pip3` to run pip3.8
92+
sudo update-alternatives --install /usr/bin/pip3 pip3 $(which pip3.8) 10
93+
# Now we can install Groundlight!
94+
pip3 install groundlight
5495
```
5596
56-
TODO: something better.
97+
## Using Groundlight on the edge
5798
58-
### Local API endpoint
99+
Starting your model evaluations at the edge reduces latency, cost, network bandwidth, and energy. Once you have downloaded and installed your Groundlight edge models, you can configure the Groundlight SDK to use your edge environment by configuring the 'endpoint' to point at your local environment as such:
59100
60-
1. Set up a local [janzu API
61-
endpoint](https://github.com/positronix-ai/zuuul/blob/main/deploy/README.md#development-using-local-microk8s)
62-
running (e.g., on an AWS GPU instance).
101+
```Python
102+
from groundlight import Groundlight
103+
gl = Groundlight(endpoint="http://localhost:6717")
104+
```
63105

64-
1. Set up an ssh tunnel to your laptop. That way, you can access the
65-
endpoint at http://localhost:8000/device-api (and the web UI at
66-
http://localhost:8000/reef):
106+
(Edge model download is not yet generally available.)
67107

68-
```Bash
69-
$ ssh instance-name -L 8000:localhost:80
70-
```
108+
## Advanced
71109

72-
1. Run the tests (with an API token)
110+
### Explicitly create a new detector
73111

74-
```Bash
75-
$ export GROUNDLIGHT_API_TOKEN=tok_abc123
76-
$ make test-local
77-
```
112+
Typically you'll use the ```get_or_create_detector(name: str, query: str)``` method to find an existing detector you've already created with the same name, or create a new one if it doesn't exists. But if you'd like to force creating a new detector you can also use the ```create_detector(name: str, query: str)``` method
78113

79-
(Note: in theory, it's possible to run the janzu API server on your
80-
laptop without microk8s - but some API methods don't work because of
81-
the dependence on GPUs)
114+
```Python
115+
detector = gl.create_detector(name="your_detector_name", query="is this what we want to see?")
116+
```
82117

83-
### Integ API endpoint
118+
### Retrieve an existing detector
84119

85-
1. Run the tests (with an API token)
120+
```Python
121+
detector = gl.get_detector(id="YOUR_DETECTOR_ID")
122+
```
86123

87-
```Bash
88-
$ export GROUNDLIGHT_API_TOKEN=tok_abc123
89-
$ make test-integ
90-
```
124+
### List your detectors
125+
126+
```Python
127+
# Defaults to 10 results per page
128+
detectors = gl.list_detectors()
129+
130+
# Pagination: 3rd page of 25 results per page
131+
detectors = gl.list_detectors(page=3, page_size=25)
132+
```
133+
134+
### Retrieve an image query
135+
136+
In practice, you may want to check for a new result on your query. For example, after a cloud reviewer labels your query. For example, you can use the `image_query.id` after the above `submit_image_query()` call.
137+
138+
```Python
139+
image_query = gl.get_image_query(id="YOUR_IMAGE_QUERY_ID")
140+
```
141+
142+
### List your previous image queries
143+
144+
```Python
145+
# Defaults to 10 results per page
146+
image_queries = gl.list_image_queries()
147+
148+
# Pagination: 3rd page of 25 results per page
149+
image_queries = gl.list_image_queries(page=3, page_size=25)
150+
```
151+
152+
### Adding labels to existing image queries
153+
154+
Groundlight lets you start using models by making queries against your very first image, but there are a few situations where you might either have an existing dataset, or you'd like to handle the escalation response programatically in your own code but still include the label to get better responses in the future. With your ```image_query``` from either ```submit_image_query()``` or ```get_image_query()``` you can add the label directly. Note that if the query is already in the escalation queue due to low ML confidence or audit thresholds, it may also receive labels from another source.
155+
156+
```Python
157+
add_label(image_query, 'YES'). # or 'NO'
158+
```
159+
160+
The only valid labels at this time are ```'YES'``` and ```'NO'```
161+
162+
163+
### Handling HTTP errors
164+
165+
If there is an HTTP error during an API call, it will raise an `ApiException`. You can access different metadata from that exception:
166+
167+
```Python
168+
from groundlight import ApiException, Groundlight
169+
170+
gl = Groundlight()
171+
try:
172+
detectors = gl.list_detectors()
173+
except ApiException as e:
174+
# Many fields available to describe the error
175+
print(e)
176+
print(e.args)
177+
print(e.body)
178+
print(e.headers)
179+
print(e.reason)
180+
print(e.status)
181+
```
91182

92-
## Releases
93-
94-
To publish a new package version to our [internal pypi
95-
repository](https://github.com/positronix-ai/packaging/tree/main/aws),
96-
you create a release on github.
97-
98-
```Bash
99-
# Create a git tag locally. Use semver "vX.Y.Z" format.
100-
$ git tag -a v0.1.2 -m "Short description"
101-
102-
# Push the tag to the github repo
103-
$ git push origin --tags
104-
```
105-
106-
Then, go to the [github
107-
repo](https://github.com/positronix-ai/groundlight-python-sdk/tags) ->
108-
choose your tag -> create a release from this tag -> type in some
109-
description -> release. A [github
110-
action](https://github.com/positronix-ai/groundlight-python-sdk/actions/workflows/publish.yaml)
111-
will trigger a release, and then `groundlight-X.Y.Z` will be available
112-
for consumers.
113-
114-
## TODOs
115-
116-
- Improve wrappers around API functions (e.g., simplify the responses even further, add auto-pagination managers, etc.)
117-
- The SDK should allow you to work with the most natural interface, rather than trying to exactly mirror the REST API.
118-
- Better auto-generated code docs (e.g. [sphinx](https://www.sphinx-doc.org/en/master/))
119-
- Model types (e.g., [autodoc_pydantic](https://github.com/mansenfranzen/autodoc_pydantic))
120-
- Cleaner auto-generated model names (e.g., `PaginatedDetectorList` is a little ugly)
121-
- Better versioning strategy. On the one hand, this package will closely follow the versioning in the HTTP API. On the other hand, we may add features in the client (like image utils, shortcuts, etc.) that are not in the REST API.
122-
- Better way of managing dependency on `public-api.yaml` OpenAPI spec (right now, we just copy the file over manually)
123-
- Update the web links (links to website, link to API endpoint, etc.)
124-
- `with` context manager (auto cleanup the client object)
125-
- It would be great to add notebooks with interactive examples that can actually run out of the box
126-
- Have a cleaner distinction between dev docs and user guide docs

0 commit comments

Comments
 (0)