API: unify sort API

[jreback]

related is #2094
related is #6847 (fixes kind and some arg ordering)
related is #7121 (make sortlevel a part of sort_index by adding level arg)

the sorting API is currently inconsistent and confusing. here is what exists:

Series:

• sort: calls Series.order, in-place, defaults quicksort
• order: do the sort on values, return a new object, defaults mergesort
• sort_index: sort by labels, returns new object

Frame:

• sort: calls sort_index
• sort_index: sorts by the index with no args, otherwise a nested sort of the passed columns

The semantics are different between Series and DataFrame. In Series, sort mean in-place, order returns a new object. sort/order sort on the values, while sort_index sorts on the index. For a DataFrame, sort and sort_index are the same and sort on a column/list of columns; inplace is a keyword.

Proposed signature of combined methods. We need to break a Series API here. because sort is an in-place method which is quite inconsistent with everything else.

def sort(self, by=None, axis=0, level=None, ascending=True, inplace=False,
                   kind='quicksort', na_last=True):

This is what I think we should do:

• make Series.sort/order be the same.
• by can take a column/list of columns (as it can now), or an index name / index to provide index sorting (which means sort by the specifiied axis)
• default is inplace=False (which is the same as now, except for Series.sort).
• Series.sort_index does s.sort('index')
• DataFrame.sort_index does df.sort('index')
• eventually deprecate Series.order
• add DataFrame.sort_columns to perform axis=1 sorting

This does switch the argument to the current sort_index, (e.g. axis is currently first), but I think then allows more natural syntax

• df.sort() or df.sort_index() or df.sort_index('index') sort on the index labels
• df.sort(['A','B'],axis=1) sort on these columns (allow 'index' here as well to sort on the index too)
• df.sort_columns() or df.sort('columns') sort on the column labels
• df.sort_columns() defaults axis=1, so df.sort_columns(['A','B']) is equiv of - - df.sort(['A','B'],axis=1)
• s.sort() sort on the values
• s.sort('index') or s.sort_index() sort on the series index

[jtratner]

What happens if you have columns named 'index' or 'columns'?

[cancan101]

Is the version of mergesort used stable? Assuming yes there are times then to prefer merge over quick.

[jreback]

mergesort is stable. the point is that right now sort and order have different defaults for the sortkind.

[cancan101]

Fwiw java uses different sorts for primitives vs objects : http://docs.oracle.com/javase/7/docs/api/java/util/Arrays.html#sort(int[], int, int)

Stability for objects speed for primitives.

[cancan101]

My point being there might actually be a valid reason to gave different defaults.

[jreback]

There IS often a valid reason to have different sorts available, that's why there is a keyword, but its incorrect to have a routine which does the same thing have different defaults.

aside from the fact that java is NOT a good example of coding :)

these are all primitives which are being sorted

[jtratner]

For 0.13 - could we change Series.sort() to return a value or add NDFrame.order() that just passes to NDFrame.sort()? It would mean that you could do the same thing no matter what kind of NDFrame you had.

[jreback]

I think we ought to fix the API - but let's think thru
prob just leave 0.13 as is

[jtratner]

okay

You can't make quicksort the default, sort stability is a contract and you can't just break it.
if you want consistency you have to default to the stricter guarantee.

numpy arrays and series currently have compatible signatures for sort:

>>> a=np.ndarray([1,1])
>>> b=pd.Series([1,2])
>>> a.sort??
a.sort(axis=-1, kind='quicksort', order=None)
In [9]: b.sort??
Definition: b.sort(self, axis=0, kind='quicksort', order=None, ascending=True)

I find it hard to believe that's a coincidence or just an act of cargo-cult when the API was designed.
You may break more then you bargained for.

In general, more "consistency" achieved by breaking existing code is not automatically a good move.
You strive for consistency when designing an API but once 10,000 people develop against that API,
consisteny has to compete with stability.

Once an API is established, the useability argument is moot since more people are used to it then not.
That just leaves aesthetic considerations, easily trumped by stability I think.

There have been too many breaking changes in recent releases and we seem to treat
it more lightly as time passes. That's a bad trend IMO.
It's not convenient or sexy to keep an API stable, but it's responsible.

TL;DR: if a 1000 people need to modify working code to accomodate this change, is it still worth doing?

[jtratner]

@y-p okay, so let's use that signature everywhere instead

You think we should drop the axis argument from df.sort()? ok, sure.