Scaffold Main Documentation using asciidoc

[asfimport]

For the general documentation of Arrow, we want to use asciidoc. The "general documentation" includes:

• The Arrow spec / memory layout
• Howtos for building arrow on different platforms
• Getting Started snippets for each language and a link to the (to-be-build) API documentation

It would be nice to have a build system and the main file/folder structure in place so we can split the work up.

Reporter: Uwe Korn / @xhochy
Assignee: Wes McKinney / @wesm

Related issues:

• Arrow 0.3 release (blocks)

_{Note: This issue was originally created as ARROW-95. Please see the migration documentation for further details.}

[asfimport]

Jacques Nadeau / @jacques-n:
I didn't see a conversation about why asciidoc versus Markdown. Did I miss the discussion? We've been using Markdown on Drill and Calcite with reasonable success.

[asfimport]

Uwe Korn / @xhochy:
There was no real discussion why asciidoc versus Markdown. @wesm suggested it and there were no objections. Probably we should raise that on the ML? At least a mail with the rough idea of how the documentation should be structured?

[asfimport]

Wes McKinney / @wesm:
We chatted briefly about it on Slack – I presumed we'd have a ML discussion or a high level JIRA about making a technical decision, so let's do that.

I'm trying to find the Drill or Calcite documentation sources, can someone send a link to those to me?

My understanding is that Markdown is much more minimal than reStructuredText or Asciidoc for creating more complex / interconnected documentation. As an example of this, named anchors don't really work in Markdown (http://stackoverflow.com/questions/6695439/how-do-you-create-link-to-a-named-anchor-in-multimarkdown), and solutions for them are HTML-specific. What if we want to generate a PDF? Maybe this need is hard to anticipate, but I'm not very comfortable using Markdown for more than READMEs and blogs (e.g. Jekyll / Pelican).

Folks with more Markdown experience making more complex documentation should comment on their experience and how much Markdown's limitations (as compared with rst / adoc) end up being problems.

[asfimport]

Wes McKinney / @wesm:
See an example of a Sphinx-generated PDF: http://pandas.pydata.org/pandas-docs/version/0.18.0/pandas.pdf. Setting aside the fact that the release notes is 250 pages, in my experience people do like having PDF versions of things.

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 589
#589