Integration/smoke tests

[dmke]

On the coming weekend In the coming days I'd like to extend the test suite with integration/smoke tests. The current suite exclusively relies on a stubbed HTTP endpoint (which is fine for local, fast tests), but doesn't actually tell me whether I can use the Gem with a specific version of InfluxDB.

Rationale

v0.11 for example introduced some subtle changes in SHOW SERIES and SHOW TAG VALUES—and although these queries are not natively covered by this Gem (yet), I would feel more confident if I can see a green smoke test result.

Battle plan

• (1) Because I'm lazy, I want a script which generates a .travis.yml from the AWS bucket (or maybe another resource) with the latest n major versions, e.g. 0.12.1, 0.11.1 and 0.10.2 but neither 0.12.0, nor 0.11.0, nor 0.10.1.

  Each version should generate an matrix.include entry:

  It should be sufficient to include the .deb download file(s) in the build matrix:

  matrix:
    include:
      - rvm: 2.3.0
        env: USE_INFLUXDB="influxdb_0.11.1-1_amd64.deb"
      - rvm: 2.3.0
        env: USE_INFLUXDB="influxdb_0.12.1-1_amd64.deb"
      - # ...

• (2) Depending on the presence of the $USE_INFLUXDB env var, a before_install or install script needs to provision the specified InfluxDB version.

• (3) The Rake default task should, in presence of ENV["USE_INFLUXDB"], only fire up the smoke test suite (maybe I can leverage script to execute rake spec:integration or sth. similar).

• (4) Finally, some actual smoke tests need to be written. As a first step, I'd transform the examples in the README.

Discussion

Did I miss something obvious? Am I crazy and shouldn't do this? Are there similar attempts in the other client library implementations (if not, should this be portable enough to act as reusable template)?

To keep my sanity, I don't really want to parse XML for (1). This might give me a nice starting point to find the relevant version prefixes, though (ex.).

/cc @toddboom

[toddboom]

@dmke This is definitely an ambitious project, but a valuable one. The Java client does something similar with Docker:
https://github.com/influxdata/influxdb-java/blob/master/src/test/java/org/influxdb/InfluxDBTest.java

And the official Go client gets used as part of the InfluxDB test suite, so there's some prior art for doing this. I'm not sure of the best way to make it portable/reusable across other libraries, so I'd say do what you think is fastest/easiest for this project in particular.

Regarding versions, I'd be careful not to go too far back - it's largely not going to be worth the effort. Based on the amount of change in the InfluxDB codebase since the v0.9.x releases, I'd start testing against v0.10.0 and not worry too much about anything prior to that.

Since there aren't all that many releases (~1-2 per month), maybe we can just manually keep a list?

[dmke]

I'm not familiar with Docker, so I'm going to stick with a plain old download & apt-get install of the .deb file.

If version 0.9.x isn't suppoted anymore, we should clarify this in the Readme (I'd expect it to work without proplems after reading the first paragraphs...) -- assuming a significant percentage still uses 0.9.x. Do you have insight and can share some datapoints from the reporting feature?

[dmke]

The basic infrastructure now exists in the smoketesting branch, starting with e2aa9a7.