datafusion-cli doesn't accept tab delimiter for CSV files

[casperhart]

Describe the bug

I would like to read a TSV file using the datafusion-cli. However, when specifying \t as the delimiter I get the error: 🤔 Invalid statement: sql parser error: Delimiter must be a single char

To Reproduce

In bash:

echo "col1\tcol2" > test.tsv
echo "1\t2" >> test.tsv

then in datafusion-cli:

create external table test stored as csv location 'test.tsv' delimiter '\t';

Expected behavior

External table creates successfully with a tab as the delimiter.

Additional context

It seems the problem is that rustyline treats the '\t' as two separate characters. I wouldn't mind having a go at fixing it myself, e.g. by updating the line from rustyline before passing to datafusion. If this is the way to do it, should all instances of \ in a given line be treated as escaped values, or only in the delimiter itself?

[r4ntix]

The current SQL parsing delimiter is treated as a string, and does not support parsing escape char.
I plan submit a PR to support it.

[alamb]

Thanks @r4ntix !