-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathdocumentation.html
More file actions
321 lines (299 loc) · 15.7 KB
/
documentation.html
File metadata and controls
321 lines (299 loc) · 15.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>LiveAPI</title>
<!-- Bootstrap -->
<link href="css/bootstrap.min.css" rel="stylesheet">
<link rel="stylesheet" href="css/font-awesome.min.css">
<link rel="stylesheet" type="text/css" href="css/isotope.css" media="screen" />
<link rel="stylesheet" href="css/animate.css">
<link rel="stylesheet" href="js/fancybox/jquery.fancybox.css" type="text/css" media="screen" />
<link href="css/style.css" rel="stylesheet">
<!-- =======================================================
Theme Name: Dewi
Theme URL: https://bootstrapmade.com/dewi-free-multi-purpose-html-template/
Author: BootstrapMade
Author URL: https://bootstrapmade.com
======================================================= -->
</head>
<body>
<header>
<nav div class="navbar navbar-default navbar-static-top" role="navigation">
<div class="container">
<ul class="social-network">
<li><a href="#" data-placement="top" title="Facebook"><i class="fa fa-facebook fa-1x"></i></a></li>
<li><a href="#" data-placement="top" title="Twitter"><i class="fa fa-twitter fa-1x"></i></a></li>
<li><a href="#" data-placement="top" title="Linkedin"><i class="fa fa-linkedin fa-1x"></i></a></li>
<li><a href="#" data-placement="top" title="Pinterest"><i class="fa fa-pinterest fa-1x"></i></a></li>
<li><a href="#" data-placement="top" title="Google plus"><i class="fa fa-google-plus fa-1x"></i></a></li>
</ul>
<ul class="info">
<!-- <li><p><i class="fa fa-phone"></i>(602) 524-5453</p></li> -->
<li><a href="#"><i class="fa fa-envelope"></i>info@liveapi.xyz</a></li>
</ul>
</div>
</nav>
<nav class="navbar navbar-default navbar-static-top" role="navigation">
<div class="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target=".navbar-collapse.collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<div class="navbar-brand">
<a href="index.html"><h1>Live<span>API</span></h1></a>
</div>
</div>
<div class="navbar-collapse collapse">
<div class="menu">
<ul class="nav nav-tabs" role="tablist">
<li role="presentation"><a href="index.html">Home</a></li>
<li role="presentation"><a href="documentation.html" class="active">Documentation</a></li>
<li role="presentation"><a href="blog.html">Blog</a></li>
<li role="presentation"><a href="screencasts.html">Screencasts</a></li>
<li role="presentation"><a href="contact.html">Contact</a></li>
</ul>
</div>
</div>
</div>
</div>
</nav>
</header>
<div class="jumbotron">
<h2>Documentation</h2>
</div>
<div class="our-services">
<div class="container">
<div class="row">
<div class="col-md-6">
<div class="wow bounceIn" data-wow-offset="0" data-wow-delay="0.4s">
<div class="services">
<div class="icons">
<a href="#local"><i class="fa fa-heart-o fa-3x"></i></a>
</div>
<h4>Local Installation</h4>
<p>
Instructions for installing the LiveAPI server on your local hardware.
</p>
</div>
</div>
<hr>
</div>
<div class="col-md-6">
<div class="wow bounceIn" data-wow-offset="0" data-wow-delay="0.8s">
<div class="services">
<div class="icons">
<a href="#local"><i class="fa fa-desktop fa-3x"></i></a>
</div>
<h4>AWS Installation</h4>
<p>
Instructions for installing the LiveAPI server on Amazon Web Services.
</p>
</div>
</div>
<hr>
</div>
<div class="col-md-6">
<div class="wow bounceIn" data-wow-offset="0" data-wow-delay="0.4s">
<div class="services">
<div class="icons">
<a href="#two"><i class="fa fa-thumbs-o-up fa-3x"></i></a>
</div>
<h4>Create Users</h4>
<p>
Create user accounts on the LiveAPI server and enable authentication.
</p>
</div>
</div>
<hr>
</div>
<div class="col-md-6">
<div class="wow bounceIn" data-wow-offset="0" data-wow-delay="0.8s">
<div class="services">
<div class="icons">
<a href="#three"><i class="fa fa-leaf fa-3x"></i></a>
</div>
<h4>Using the Chrome Extension</h4>
<p>
DOM element selection, user authentication, endpoint and interval creation.
</p>
</div>
</div>
<hr>
</div>
</div>
</div>
<div class="container">
<h1 id="local">Using LiveAPI Part 1: Installation</h1>
Successful apps are built on data. As developers, we don’t always have access to the data that would help make our app successful. While the internet is a nearly-bottomless source of public data in the form of websites, that data is not always structured or available programmatically through an API. Time spent building an extraction algorithm and server is time not spent building your app.
We’re developing LiveAPI, a developer tool to turn any website’s public data into an API in a few minutes. LiveAPI has two parts: a Chrome Extension to select data to extract and a user-hostable server that extracts data and serves up the user-created API endpoints.
<br><br>
This is the first in a three-part guide that walks through how to get started and use LiveAPI.
<br><br>
Part 1: Installation
<br>
Part 2: Authentication
<br>
Part 3: Using the Chrome Extension
<br><br>
<h3>Installation Guide</h3>
A primary goal of ours when building LiveAPI is to make setup painless. Installing the Chrome Extension only takes two clicks on the Chrome Web Store. You can now start using the LiveAPI extension on any site by clicking the icon in the top-right Chrome Menu and following the Using the Chrome Extension part of this guide.
Setting up a server generally isn’t as simple. Keeping to our goal, we’ve automated the server installation process through a script that installs all of the necessary prerequisites, builds the code and starts the LiveAPI services. This script can be executed in one shell command:
<br><br>
<i>sudo curl -shttps://raw.githubusercontent.com/live-API/LAPI-Server/master/bin/pull.sh | bash -s </i>
<br><br>
This command is tested for MacOS and Amazon Linux, but should work on other flavors of RHEL and on Ubuntu. If you have any setup or compatibility issues, let us know on Github. We’re still early in determining our compatibility and would love your feedback.
<br><br>
<h3>Port Setup</h3>
LiveAPI server runs on port 4000 (http) and 4443 (https) by default and is accessible by adding the port to the domain (e.g. liveapi.yoursite.com:4443). To serve and access LiveAPI over the standard http and https ports, you can update the HTTP_PORT and HTTPS_PORT variables in server.js.
<br><br>
If using AWS to host LiveAPI, EC2 requires the application to have root access to run on ports under 1000. To avoid running a web server like LiveAPI as root, you can use iptables to forward port 443 to LiveAPI’s 4443. See this guide for more information.
What We’re Asking to Sudo
<br><br>
The setup command grabs a shell script from github and runs it as the root. Since we don’t recommend executing unfamiliar code as root, we’ll outline what the scripts do.
The pull.sh script is relatively simple and does the following:<br><br>
Installs git<br>
Pulls the latest LiveAPI server code from github<br>
Starts the installation script (start.sh)
<br><br>
The installation script, start.sh, contains everything else need to install and start the server:<br>
Installs NodeJS/npm<br>
Installs MongoDB<br>
Installs Node module dependencies<br>
Builds and bundles the front-end JS<br>
Generates self-signed SSL certificates<br>
Starts the server
<br><br>
Starting the server using npm start does the following:<br>
Starts the Mongo daemon<br>
Starts LiveAPI<br>
Manual Installation
<br><br>
Alternatively, LiveAPI can be installed and started manually. Make sure to do each of the steps listed in the bullets above.
If you are bringing your own SSL certs, such as from a CA, you can drop the certificate (cert.pem) and private key (key.pem) in the SSL folder in the root of LiveAPI.
<br><br>
The minimum NodeJS version required for LiveAPI is 7, but there may be other requirements we have not found yet. We have tested on NodeJS >8.1.4 and MongoDB >3.4.4, but let us know on Github if you have any problems with these or other versions.
<br><br>
<h1 id="two">Using LiveAPI Part 2: Creating Users and Authentication</h1>
If you have not installed the LiveAPI server on your computer, or a cloud-hosting solution like AWS, read Part 1 to setup the server.
<br><br>
By now, here is what you have completed:
<br>
[x] Installation on AWS / Locally
<br>
[x] Setup SSL Certificate
<br>
[x] Start the LiveAPI server
<br><br>
We built authorization into LiveAPI to allow you to control the endpoints created. First, create the admin account.
<br>
<h3>Creating an Admin Account</h3>
Go to https://localhost:4443/config. If your SSL Certificate is working correctly, you will see the above sign-in bar. Create the admin account by entering your login credentials. This will create a cookie in the browser, and bring you to the new page.
<br><br>
If you get an error, your SSL certificate is not working. Visit Part 1: Installation on instructions to create an SSL certificate!
<br><br>
<h3>Invite additional users</h3>
When you want multiple people accessing the created endpoints, use invitation links to create their account. This gives the admin control over who has access to the endpoints created, and the corresponding data that is scraped.Entering the invitation link will prompt a username and password.
<br><br>
<h1 id="three">Using LiveAPI Part 3: Chrome Extension</h1>
If you have not installed the LiveAPI server on your computer, or a cloud-hosting solution like AWS, read Part 1 to setup the server.
<br><br>
If you have not created one or more user accounts on your server, read Part 2 on authentication.
<br><br>
Once steps one and two are finished, install the LiveAPI Chrome Extension from the Chrome Extension webstore.To activate the extension, navigate to the target URL you wish to scrape and click the browser action icon (right side on browser top bar). The UI bar will move you through the necessary steps for designing your endpoint.
<br><br>
<h3>Step 1:</h3> Click on a DOM element of your choice that contains one of the properties that you want in your data objects; you will see your selection highlighted in yellow. Select another of the same kind on the page and if the elements are structurally similar, LiveAPI will highlight the remaining items of that kind on the page. Deselect anything you do not wish to include. Enter a name for your new property (such as “title”) then click “Save”. Repeat as desired for additional elements.
<br><br>
<h3>Note:</h3> Use names with no spaces that are camelCased or snake_cased. Please do not use spaces or special characters besides dashes. These will be your keys in your key value pairs of each object.
<br><br>
Step 2 and 3 are not functional at the moment and are in development.
<br><br>
<h3>Step 4:</h3> Authenticate yourself with the credentials you created on the LiveAPI server. The address field is where you indicate the URL and port of your server. In most cases the port will be 4443.
<br><br>
<h3>Step 5:</h3> Create a name for your endpoint. Observe the same rules for naming as above because this name will be part of your endpoint URI. Enter an interval measured in seconds and click “Create Endpoint”. You will then have the choice to navigate to your endpoint to verify accuracy of your data or you can elect to begin another endpoint design.
<br><br>
And that’s it!
</div>
</div>
<div class="jumbotron">
<h1>Because data should be<span> open and free.</span> </h1>
<p>LiveAPI is and always will be open source. We welcome contributions from other developers.</p>
<p><a class="btn btn-primary btn-lg" href="https://github.com/Live-API" role="button">Go to Github</a></p>
</div>
<footer>
<div class="inner-footer">
<div class="container">
<div class="row">
<div class="col-md-4 f-about">
<a href="index.html"><h1>Live<span>API</span></h1></a>
<p>LiveAPI offers elegant, extensible and easy to use data extraction with the touch of a button (or rather, a short bash command). Serve your own APIs locally or on an AWS server; simply set up the server, the corresponding LiveAPI Chrome Extension and painlessly design your own JSON data to serve. </p>
</div>
<div class="col-md-4 l-posts">
<h3 class="widgetheading">Latest Posts</h3>
<ul>
<li><a href="documentation.html">Get started</a></li>
<li><a href="blog.html">Data Extraction Challenges in the Modern Web</a></li>
<li><a href="index.html">About the Team</a></li>
<li><a href="index.html">Links</a></li>
</ul>
</div>
<div class="col-md-4 f-contact">
<h3 class="widgetheading">Stay in touch</h3>
<a href="#"><p><i class="fa fa-envelope"></i> info@liveapi.xyz</p></a>
<!-- <p><i class="fa fa-phone"></i> (000) 524-5453</p> -->
<p><i class="fa fa-home"></i> LiveAPI Limited<br>
5300 Beethoven Street<br>
PH Floor (4th), Los Angeles <br>
California 90025 USA</p>
</div>
</div>
</div>
</div>
<div class="last-div">
<div class="container">
<div class="row">
<div class="copyright">
© Dewi Theme. All Rights Reserved
<div class="credits">
<!--
All the links in the footer should remain intact.
You can delete the links only if you purchased the pro version.
Licensing information: https://bootstrapmade.com/license/
Purchase the pro version with working PHP/AJAX contact form: https://bootstrapmade.com/buy/?theme=Dewi
-->
<a href="https://bootstrapmade.com/free-business-bootstrap-themes-website-templates/">Business Bootstrap Themes</a> by <a href="https://bootstrapmade.com/">BootstrapMade</a>
</div>
</div>
<nav class="foot-nav">
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="documentation.html">Documentation</a></li>
<li><a href="blog.html">Blog</a></li>
<li><a href="screencasts.html">Screencasts</a></li>
<li><a href="contact.html">Contact</a></li>
</ul>
</nav>
<div class="clear"></div>
</div>
</div>
<a href="" class="scrollup"><i class="fa fa-chevron-up"></i></a>
</div>
</footer>
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<script src="js/jquery-2.1.1.min.js"></script>
<!-- Include all compiled plugins (below), or include individual files as needed -->
<script src="js/bootstrap.min.js"></script>
<script src="js/wow.min.js"></script>
<script src="js/fancybox/jquery.fancybox.pack.js"></script>
<script src="js/jquery.easing.1.3.js"></script>
<script src="js/jquery.bxslider.min.js"></script>
<script src="js/functions.js"></script>
<script>wow = new WOW({}).init();</script>
</body>
</html>