Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
## HEAD (unreleased)

## 1.0.0

- Gem name changed from `syntax_search` to `dead_end` (https://github.com/zombocom/syntax_search/pull/30)
- Moved `syntax_search/auto` behavior to top level require (https://github.com/zombocom/syntax_search/pull/30)
- Error banner now indicates when missing a `|` or `}` in addition to `end` (https://github.com/zombocom/syntax_search/pull/29)
- Trailing slashes are now handled (joined) before the code search (https://github.com/zombocom/syntax_search/pull/28)

Expand Down
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

source "https://rubygems.org"

# Specify your gem's dependencies in syntax_search.gemspec
# Specify your gem's dependencies in dead_end.gemspec
gemspec

gem "rake", "~> 12.0"
Expand Down
4 changes: 2 additions & 2 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
PATH
remote: .
specs:
syntax_search (0.2.0)
dead_end (1.0.0)

GEM
remote: https://rubygems.org/
Expand All @@ -27,10 +27,10 @@ PLATFORMS
ruby

DEPENDENCIES
dead_end!
rake (~> 12.0)
rspec (~> 3.0)
stackprof
syntax_search!

BUNDLED WITH
2.1.4
87 changes: 46 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,27 @@
# SyntaxSearch
# DeadEnd

Imagine you're programming, everything is awesome. You write code, it runs. Still awesome. You write more code, you save it, you run it and then:
An AI powered library to find syntax errors in your source code:

```
file.rb:333: syntax error, unexpected `end', expecting end-of-input
```

What happened? Likely you forgot a `def`, `do`, or maybe you deleted some code and missed an `end`. Either way it's an extremely unhelpful error and a very time consuming mistake to track down.

What if I told you, that there was a library that helped find your missing `def`s and missing `do`s. What if instead of searching through hundreds of lines of source for the cause of your syntax error, there was a way to highlight just code in the file that contained syntax errors.

$ syntax_search path/to/file.rb

SyntaxSearch: Unmatched `end` detected
DeadEnd: Unmatched `end` detected

This code has an unmatched `end`. Ensure that all `end` lines
in your code have a matching syntax keyword (`def`, `do`, etc.)
and that you don't have any extra `end` lines.

file: path/to/file.rb
file: path/to/dog.rb
simplified:

1 require 'zoo'
2
3 class Animal
4
❯ 5 defdog
3 class Dog
❯ 5 defbark
❯ 7 end
8
12 end

How much would you pay for such a library? A million, a billion, a trillion? Well friends, today is your lucky day because you can use this library today for free!

## Installation in your codebase

To automatically search syntax errors when they happen, add this to your Gemfile:

```ruby
gem 'syntax_search', require: "syntax_search/auto"
gem 'dead_end'
```

And then execute:
Expand All @@ -47,13 +31,13 @@ And then execute:
If your application is not calling `Bundler.require` then you must manually add a require:

```ruby
require "syntax_search/auto"
require "dead_end"
```

If you're using rspec add this to your `.rspec` file:

```
--require syntax_search/auto
--require dead_end
```

> This is needed because people can execute a single test file via `bundle exec rspec path/to/file_spec.rb` and if that file has a syntax error, it won't load `spec_helper.rb` to trigger any requires.
Expand All @@ -62,13 +46,40 @@ If you're using rspec add this to your `.rspec` file:

To get the CLI and manually search for syntax errors, install the gem:

$ gem install syntax_search
$ gem install dead_end

This gives you the CLI command `$ dead_end` for more info run `$ dead_end --help`.

## What syntax errors does it handle?

- Missing `end`:

```
class Dog
def bark
puts "bark"

def woof
puts "woof"
end
end
# => scratch.rb:8: syntax error, unexpected end-of-input, expecting `end'
```

This gives you the CLI command `$ syntax_search` for more info run `$ syntax_search --help`.
- Unexpected `end`

## What does it do?
```
class Dog
def speak
@sounds.each |sound| # Note the missing `do` here
puts sound
end
end
end
# => scratch.rb:7: syntax error, unexpected `end', expecting end-of-input
```

When your code triggers a SyntaxError due to an "unexpected `end'" in a file, this library fires to narrow down your search to the most likely offending locations.
As well as unmatched `|` and unmatched `}`. These errors can be time consuming to debug because Ruby often only tells you the last line in the file. The command `ruby -wc path/to/file.rb` can narrow it down a little bit, but this library does a better job.

## Sounds cool, but why isn't this baked into Ruby directly?

Expand All @@ -77,6 +88,10 @@ I would love to get something like this directly in Ruby, but I first need to pr
1. Get real world useage and feedback. If we gave you an awful suggestion, let us know! We try to handle lots of cases well, but maybe we could be better.
2. Prove out demand. If you like this idea, then vote for it by putting it in your Gemfile.

## Artificial Inteligence?

This library uses a goal-seeking algorithm similar to that of a path-finding search. For more information [read the blog post about how it works under the hood](https://schneems.com/2020/12/01/squash-unexpectedend-errors-with-syntaxsearch/).

## How does it detect syntax error locations?

We know that source code that does not contain a syntax error can be parsed. We also know that code with a syntax error contains both valid code and invalid code. If you remove the invalid code, then we can programatically determine that the code we removed contained a syntax error. We can do this detection by generating small code blocks and searching for which blocks need to be removed to generate valid source code.
Expand All @@ -87,16 +102,6 @@ Here's an example:

![](assets/syntax_search.gif)

## How is source code broken up into smaller blocks?

By definition source code with a syntax error in it cannot be parsed, so we have to guess how to chunk up the file into smaller pieces. Once we've split up the file we can safely rule out or zoom into a specific piece of code to determine the location of the syntax error. This libary uses indentation and empty lines to make guesses about what might be a "block" of code. Once we've got a chunk of code, we can test it.

At the end of the day we can't say where the syntax error is FOR SURE, but we can get pretty close. It sounds simple when spelled out like this, but it's a very complicated problem. Even when code is not correctly indented/formatted we can still likely tell you where to start searching even if we can't point at the exact problem line or location.

## How does this gem know when a syntax error occured in my code?

Right now the search isn't performed automatically when you get a syntax error. Instead we append a warning message letting you know how to test the file. Eventually we'll enable the seach by default instead of printing a warning message. To do both of these we have to monkeypatch `require` in the same way that bootsnap does.

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
Expand All @@ -105,7 +110,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/zombocom/syntax_search. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/syntax_search/blob/master/CODE_OF_CONDUCT.md).
Bug reports and pull requests are welcome on GitHub at https://github.com/zombocom/dead_end. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/zombocom/dead_end/blob/master/CODE_OF_CONDUCT.md).


## License
Expand All @@ -114,4 +119,4 @@ The gem is available as open source under the terms of the [MIT License](https:/

## Code of Conduct

Everyone interacting in the SyntaxErrorSearch project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/zombocom/syntax_search/blob/master/CODE_OF_CONDUCT.md).
Everyone interacting in the DeadEnd project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/zombocom/dead_end/blob/master/CODE_OF_CONDUCT.md).
2 changes: 1 addition & 1 deletion bin/console
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env ruby

require "bundler/setup"
require "syntax_search"
require "dead_end"

# You can add fixtures and/or initialization code here to make experimenting
# with your gem easier. You can also use a different console, if you like.
Expand Down
10 changes: 5 additions & 5 deletions syntax_search.gemspec → dead_end.gemspec
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# frozen_string_literal: true

require_relative 'lib/syntax_search/version'
require_relative 'lib/dead_end/version'

Gem::Specification.new do |spec|
spec.name = "syntax_search"
spec.version = SyntaxErrorSearch::VERSION
spec.name = "dead_end"
spec.version = DeadEnd::VERSION
spec.authors = ["schneems"]
spec.email = ["richard.schneeman+foo@gmail.com"]

spec.summary = %q{Find syntax errors in your source in a snap}
spec.description = %q{When you get an "unexpected end" in your syntax this gem helps you find it}
spec.homepage = "https://github.com/zombocom/syntax_search.git"
spec.homepage = "https://github.com/zombocom/dead_end.git"
spec.license = "MIT"
spec.required_ruby_version = Gem::Requirement.new(">= 2.5.0")

spec.metadata["homepage_uri"] = spec.homepage
spec.metadata["source_code_uri"] = "https://github.com/zombocom/syntax_search.git"
spec.metadata["source_code_uri"] = "https://github.com/zombocom/dead_end.git"

# Specify which files should be added to the gem when it is released.
# The `git ls-files -z` loads the files in the RubyGem that have been added into git.
Expand Down
30 changes: 13 additions & 17 deletions exe/syntax_search → exe/dead_end
Original file line number Diff line number Diff line change
Expand Up @@ -2,39 +2,35 @@

require 'pathname'
require "optparse"
require_relative "../lib/syntax_search.rb"
require_relative "../lib/dead_end.rb"

options = {}
options[:terminal] = true
options[:record_dir] = ENV["SYNTAX_SEARCH_RECORD_DIR"]
options[:record_dir] = ENV["DEAD_END_RECORD_DIR"]

parser = OptionParser.new do |opts|
opts.banner = <<~EOM
Usage: syntax_search <file> [options]
Usage: dead_end <file> [options]

Parses a ruby source file and searches for syntax error(s) unexpected `end', expecting end-of-input.
Parses a ruby source file and searches for syntax error(s) such as
unexpected `end', expecting end-of-input.

Example:

$ syntax_search dog.rb
$ dead_end dog.rb

# ...

```
1 require 'animals'
2
❯ 10 defdog
❯ 15 end
❯ 16
20 def cat
22 end
```
❯ 10 defdog
❯ 15 end
❯ 16

Env options:

SYNTAX_SEARCH_RECORD_DIR=<dir>
DEAD_END_RECORD_DIR=<dir>

When enabled, records the steps used to search for a syntax error to the given directory
When enabled, records the steps used to search for a syntax error to the
given directory

Options:
EOM
Expand Down Expand Up @@ -66,7 +62,7 @@ options[:record_dir] = "tmp" if ENV["DEBUG"]

$stderr.puts "Record dir: #{options[:record_dir]}" if options[:record_dir]

SyntaxErrorSearch.call(
DeadEnd.call(
source: file.read,
filename: file.expand_path,
terminal: options[:terminal],
Expand Down
4 changes: 4 additions & 0 deletions lib/dead_end.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# frozen_string_literal: true

require_relative "dead_end/internals"
require_relative "dead_end/auto"
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true
#
module SyntaxErrorSearch
module DeadEnd
# This class is useful for exploring contents before and after
# a block
#
Expand Down
12 changes: 6 additions & 6 deletions lib/syntax_search/auto.rb → lib/dead_end/auto.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
require_relative "../syntax_search"
require_relative "../dead_end/internals"

# Monkey patch kernel to ensure that all `require` calls call the same
# method
Expand All @@ -10,13 +10,13 @@ module Kernel
def load(file, wrap = false)
original_load(file)
rescue SyntaxError => e
SyntaxErrorSearch.handle_error(e)
DeadEnd.handle_error(e)
end

def require(file)
original_require(file)
rescue SyntaxError => e
SyntaxErrorSearch.handle_error(e)
DeadEnd.handle_error(e)
end

def require_relative(file)
Expand All @@ -26,7 +26,7 @@ def require_relative(file)
original_require File.expand_path("../#{file}", caller_locations(1, 1)[0].absolute_path)
end
rescue SyntaxError => e
SyntaxErrorSearch.handle_error(e)
DeadEnd.handle_error(e)
end
end

Expand All @@ -39,13 +39,13 @@ class Object
def load(path, wrap = false)
Kernel.load(path, wrap)
rescue SyntaxError => e
SyntaxErrorSearch.handle_error(e)
DeadEnd.handle_error(e)
end

def require(path)
Kernel.require(path)
rescue SyntaxError => e
SyntaxErrorSearch.handle_error(e)
DeadEnd.handle_error(e)
end
end

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# frozen_string_literal: true
module SyntaxErrorSearch
module DeadEnd
# This class is responsible for taking a code block that exists
# at a far indentaion and then iteratively increasing the block
# so that it captures everything within the same indentation block.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

module SyntaxErrorSearch
module DeadEnd

# Given a block, this method will capture surrounding
# code to give the user more context for the location of
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

module SyntaxErrorSearch
module DeadEnd
# Multiple lines form a singular CodeBlock
#
# Source code is made of multiple CodeBlocks.
Expand Down Expand Up @@ -68,7 +68,7 @@ def invalid?
end

def valid?
SyntaxErrorSearch.valid?(self.to_s)
DeadEnd.valid?(self.to_s)
end

def to_s
Expand Down
Loading