polyglot update

.circleci/config.yml

@@ -58,7 +58,7 @@ jobs:
       - *git_restore_cache
       - *mvn_restore_cache
       - run:
-          lein test
+          cd clj_clojure && lein test
 
 workflows:
   version: 2

doc/poly.md

@@ -0,0 +1,156 @@
+# Jukebox - Polyglot Update
+
+This update makes jukebox work with steps defined in multiple languages. Each step can be defined in a supported language (currently clojure or ruby). When a scenario is run, each step will be run in whichever language it's defined with. 
+
+## Syntax
+Here are some example step definition snippets:
+
+    ჻ clojure -A:jukebox/snippets test/features
+    UUU
+    Undefined scenarios:
+    test/features/belly.feature:3 # a few cukes
+
+    1 Scenarios (1 undefined)
+    3 Steps (3 undefined)
+    0m3.145s
+
+
+    You can implement missing steps with the snippets below:
+
+      ```clojure
+      (defn i-have-cukes-in-my-belly
+        "Returns an updated context (`board`)."
+        {:scene/step "I have {int} cukes in my belly"}
+        [board, int1]
+        ;; Write code here that turns the phrase above into concrete actions
+        (throw (cucumber.api.PendingException.))
+        board) ;; Return the board
+      ```
+
+      ```ruby
+      require jukebox
+      module MyTests
+        extend Jukebox
+
+        step 'I have {int} cukes in my belly' do |board, int1|
+          pending! # Write code here that turns the phrase above into concrete actions
+          board # return the updated board
+        end
+      end
+      ```
+
+## How it works (architecture)
+1. When jukebox starts up, it starts a coordinator thread. The coordinator will launch jukebox language clients, one for each language. The coordinator sits between cucumber (which will parse feature files written in gherkin) and the language clients, dispatching requests (from cucumber) to run a step to the appropriate language client. 
+2. In the default configuration, the coordinator will determine what languages are active. Currently, clojure is detected by the presence of a `deps.edn` or `project.clj` file in the current directory. Ruby is detected with the presense of a `Gemfile`. This can also be defined explicitly in a `.jukebox` file (see below).
+3. A `jukebox language client` is launched for each language. In the default configuration, the clojure client is launched in memory. The ruby client is launched by running `bundle exec jcl_ruby`. This can explicitly configured in a `.jukebox` file.
+4. In the current iteration, communication between the language clients and coordinator happens via a JSON-formatted protocol over a websocket connection. When the coordinator starts up, it creates a websocket server on a random port. When it launches each language client, the port is provided.
+5. At launch, a language client will scan the feature paths for step definitions, creating an internal registry. It will then connect to the jukebox coordinator via a websocket client, and provide it's step definition inventory to the coordinator.
+6. As the features are executed by cucumber, the coordinator will look up which client knows how to handle a step, and dispatch the request to the right client.
+
+## Configuration
+A jukebox configuration can be defined explicitly if needed in a project by creating a `.jukebox` file with the following (JSON):
+```json
+{"languages": ["ruby", "clojure"]}
+``` 
+
+In addition, the launcher details can be configured if needed by adding the "language-clients" configuration. These are the defaults:
+```json
+{"languages": ["ruby", "clojure"],
+ "language-clients": [{"language": "clojure", "launcher": "jlc-clj-embedded"},
+                      {"language": "ruby", "launcher": "jlc-cli", "cmd": ["bundle", "exec", "jlc_ruby"]}]}
+```
+
+## Ruby Details
+### Defining Step Definitions & Hooks
+Step definitions can be defined by requiring 'jukebox' and using `step`:
+
+```ruby
+require jukebox
+
+module MyTests
+  extend Jukebox # Mixin `Jukebox.step` so it can be used as `step`
+
+  step 'I have {int} cukes in my belly' do |board, int1|
+    pending! # Write code here that turns the phrase above into concrete actions
+    board # return the updated board
+  end
+
+  step :before do |board scenario|
+    pending! # Write code here that runs before each scenario
+    board # return the updated board
+  end
+
+  step :before {:tags "@user and @admin"} do |board|
+      pending! # Write code here that will run before each scenario that matches the tag expression
+      board # return the updated board
+  end
+
+  step :after do |board scenario|
+    pending! # Write code here that runs after each scenario
+    board # return the updated board
+  end
+
+  step :after_step do |board scenario|
+    pending! # Write code here that runs before each step
+    board # return the updated board
+  end
+
+  step :before_step do |board scenario|
+    pending! # Write code here that runs after each step
+    board # return the updated board
+  end
+end
+```
+
+### Cucumber Compatibility
+If a step is defined in a cucumber style (`When`, `Then`, etc), then the Ruby jukebox language client will switch to Cucumber compatibility mode. This mode replicates / requires the code to be laid out in the cucumber conventions. In compatibility mode, the `board` is not provided to the step definition, unless the arity supports it.
+
+## Clojure Details
+### Defining steps with metadata tags
+Functions can be tagged as step definitions using function meta:
+
+```clojure
+(defn i-have-cukes-in-my-belly
+  "Returns an updated context (`board`)."
+  {:scene/step "I have {int} cukes in my belly"}
+  [board, int1]
+  ;; Write code here that turns the phrase above into concrete actions
+  (throw (cucumber.api.PendingException.))
+  board) ;; Return the board
+```
+
+Functions can be tagged as hooks with the metadata keys: `:step/before`, `:step/after`, `:step/before-step`, or `:step/after-step`:
+```clojure
+(defn ^:scene/before webdriver-initialize
+  "Initialize a webdriver."
+  [board scenario]
+  (assoc board :web-driver (web/driver)))
+```
+
+### Defining steps with the `step` macro
+Steps can now alternatively be defined with the `step` macro that works like the Ruby version:
+
+```clojure
+(ns example.belly
+  (:require [fundingcircle.jukebox :refer [step]]))
+
+(step "I have {int} cukes in my belly"
+  [board int1]
+  board) ;; return the updated board
+
+(step :before ;; Run before every scenario
+  [board scenario]
+  board)
+
+(step :before-step {:tags "@user and @admin"} ;; Run before the scenarios with the matching tags
+  [board scenario]
+  board)
+
+(step :after ;; Run after each scenario
+  [board scenario]
+  board)
+
+(step :after-step ;; Run after each step
+  [board scenario]
+  board)       
+```

jlc_clojure/.gitignore

@@ -0,0 +1,15 @@
+/target
+/classes
+/checkouts
+profiles.clj
+pom.xml
+pom.xml.asc
+*.jar
+*.class
+/.lein-*
+/.nrepl-port
+.hgignore
+.hg/
+.cpcache
+cucumber.json
+Gemfile.lock

jlc_clojure/.ruby-version

@@ -0,0 +1 @@
+2.6.5

jlc_clojure/Gemfile

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+gem 'jukebox', path: '../jlc_ruby', group: :development

jlc_clojure/project.clj

@@ -0,0 +1,27 @@
+(defproject fundingcircle/jukebox "1.0.5-SNAPSHOT"
+  :description "A clojure BDD library that integrates with cucumber."
+  :url "https://github.com/fundingcircle/jukebox/"
+  :license {:name "BSD 3-clause"
+            :url "http://opensource.org/licenses/BSD-3-Clause"}
+  :dependencies [[cheshire "5.8.1"]
+                 [clojure-msgpack "1.2.1"]
+                 [io.cucumber/cucumber-core "4.7.1"]
+                 [io.cucumber/cucumber-junit "4.7.1"]
+                 [me.raynes/conch "0.8.0"]
+                 [org.clojure/clojure "1.10.1"]
+                 [org.clojure/core.async "0.4.500"]
+                 [org.clojure/tools.cli "0.4.2"]
+                 [org.clojure/tools.logging "0.4.1"]
+                 [org.clojure/tools.namespace "0.2.11"]
+                 [venantius/yagni "0.1.7"]]
+  :profiles {:dev {:aliases {"cucumber" ["run" "-m" "cucumber.api.cli.Main"
+                                         "--glue" "test/example"
+                                         "--plugin" "json:cucumber.json"
+                                         "--plugin" "pretty"
+                                         "test/features"]
+                             "inventory" ["run" "-m" "fundingcircle.jukebox.alias.inventory" "--glue" "test/example"]}
+                   :source-paths ["src" "junit"]
+                   :dependencies [[ch.qos.logback/logback-classic "1.2.3"]
+                                  [net.mikera/cljunit "0.7.0"]]
+                   :resource-paths ["test"]}}
+  :aot [fundingcircle.jukebox.backend.cucumber])

jlc_clojure/src/fundingcircle/jukebox.clj

@@ -0,0 +1,53 @@
+(ns fundingcircle.jukebox
+  "Defines the Jukebox DSL."
+  (:require [fundingcircle.jukebox.step-registry :as step-registry]
+            [clojure.string :as str])
+  (:import (java.util UUID)
+           (cucumber.api PendingException)))
+
+(def ^:private trigger?
+  "Checks whether a value is a string or keyword."
+  (some-fn keyword? string?))
+
+(defn- step-fn-name
+  "Returns a name for a step."
+  [trigger]
+  (symbol
+    (if (keyword? trigger)
+      (str (name trigger) "-" (UUID/randomUUID))
+      (str/lower-case
+        (str/join "-" (remove str/blank? (str/split trigger #"\W")))))))
+
+(defmacro step
+  "Defines a step.
+
+  Examples:
+    ;; Define a step
+    (step \"I have {int} cukes in my belly\"
+      [board number-of-cukes]
+      board)
+
+    ;; Run before scenarios with tags:
+    (step :before {:tags \"@foo or @bar\"}
+      [board scenario]
+      board)"
+  {:style/indent 1}
+  [& triggers-opts-args-body]
+  (let [triggers  (take-while trigger? triggers-opts-args-body)
+        opts_body (drop-while trigger? triggers-opts-args-body)
+        opts      (first (take-while map? opts_body))
+        args_body (drop-while map? opts_body)
+        args      (first (take-while vector? args_body))
+        body      (drop-while vector? args_body)]
+    `(do
+       ~@(for [trigger triggers]
+           `(defn ~(step-fn-name trigger)
+              ~(cond-> (assoc opts :scene/step trigger) (:tags opts)
+                       (assoc :scene/tags (:tags opts)))
+              ~args
+              ~@body)))))
+
+(defn pending!
+  "Marks a step definition as pending."
+  []
+  (throw (PendingException.)))

jlc_clojure/src/fundingcircle/jukebox/alias/client.clj

@@ -0,0 +1,61 @@
+(ns fundingcircle.jukebox.alias.client
+  "Entry point for the jukebox language client for clojure."
+  (:require [clojure.edn :as edn]
+            [clojure.java.io :as io]
+            [clojure.string :as str]
+            [clojure.tools.cli :as cli]
+            [fundingcircle.jukebox.client :as client]))
+
+(defn- slurp-deps
+  "Loads `deps.edn` in the current working directory."
+  []
+  (let [deps (io/file "deps.edn")]
+    (when (.exists deps)
+      (edn/read-string (slurp deps)))))
+
+(defn- glue-paths
+  "Returns the list of glue paths.
+
+  - If paths are provided in args as `-g` or `--glue` options, those are used
+  - Otherwise, `:paths` in `deps.edn`
+  - Otherwise, `[\"src\", \"test\", \"src/main/clojure\", \"src/test/clojure\"]`"
+  [glue_paths]
+  (or (seq glue_paths)
+      (:paths (slurp-deps))
+      ["src" "test" "src/main/clojure" "src/test/clojure"]))
+
+
+(def ^:private cli-options
+  "Command line options."
+  [["-p" "--port PORT" "Port number"
+    :parse-fn #(Integer/parseInt %)
+    :validate [#(< 0 % 0x65536) "Invalid port number"]]
+   ["-h" "--help" "Prints this help"]])
+
+(defn- banner
+  "Print the command line banner."
+  [summary]
+  (println "Usage: jlc_clojure [options] <glue paths>.\n%s")
+  (println summary))
+
+(defn -main
+  "Launch the clojure jukebox language client from the command line."
+  [& args]
+  (let [{:keys [options arguments errors summary]} (cli/parse-opts args cli-options)]
+    (cond
+      (:help options)
+      (banner summary)
+
+      (not (:port options))
+      (banner summary)
+
+      errors
+      (do
+        (binding [*out* *err*] (println (str/join \newline errors)))
+        (System/exit 1))
+
+      :else (try
+              (client/start nil (:port options) (glue-paths arguments))
+              (catch Throwable e
+                (prn e)
+                (.printStackTrace e))))))

src/fundingcircle/jukebox/alias/cucumber.clj → jlc_clojure/src/fundingcircle/jukebox/alias/cucumber.clj

@@ -1,8 +1,7 @@
 (ns fundingcircle.jukebox.alias.cucumber
   "Entry point for cucumber command."
   (:require [clojure.edn :as edn]
-            [clojure.java.io :as io])
-  (:import cucumber.api.cli.Main))
+            [clojure.java.io :as io]))
 
 (defn- slurp-deps
   "Loads `deps.edn` in the current working directory."
@@ -22,10 +21,11 @@
     (->>
      (interleave
       (repeat "--glue")
-      (or (:paths (slurp-deps))
+      (or (map #(.getAbsolutePath (io/file %)) (:paths (slurp-deps)))
+        #_(:paths (slurp-deps))
           ["src" "test" "src/main/clojure" "src/test/clojure"]))
      (into []))))
 
 (defn -main [& args]
-  (cucumber.api.cli.Main/main
-   (into-array String (concat (glue-paths args) args))))
+  (io.cucumber.core.cli.Main/main
+   (into-array String (concat ["file:."] args) #_ (concat (glue-paths args) args))))

jlc_clojure/src/fundingcircle/jukebox/alias/inventory.clj

@@ -0,0 +1,33 @@
+(ns fundingcircle.jukebox.alias.inventory
+  "Entry point for generating an inventory of steps tagged with`:scene/resources`."
+  (:require [clojure.edn :as edn]
+            [clojure.java.io :as io]
+            [fundingcircle.jukebox.coordinator :as coordinator]))
+
+(defn- slurp-deps
+  "Loads `deps.edn` in the current working directory."
+  []
+  (let [deps (io/file "deps.edn")]
+    (when (.exists deps)
+      (edn/read-string (slurp deps)))))
+
+(defn- glue-paths
+  "Returns the list of glue paths.
+
+  - If paths are provided in args as `-g` or `--glue` options, those are used
+  - Otherwise, `:paths` in `deps.edn`
+  - Otherwise, `[\"src\", \"test\", \"src/main/clojure\", \"src/test/clojure\"]`"
+  [args]
+  (if (some #{"-g" "--glue"} args)
+    (mapv second (partition 2 args))
+    (->>
+      (or (:paths (slurp-deps))
+          ["src" "test" "src/main/clojure" "src/test/clojure"])
+      (into []))))
+
+(defn -main [& args]
+  (let [{:keys [resources]} (coordinator/start (glue-paths args))]
+    (coordinator/stop)
+    (doseq [resource resources]
+      (println resource))
+    (System/exit 0)))                                       ;; TODO: Fix