Merge PR #3663

Author: wesm

RELEASE.rst

@@ -40,8 +40,8 @@ pandas 0.11.1
     list of the rows from which to read the index. Added the option,
     ``tupleize_cols`` to provide compatiblity for the pre 0.11.1 behavior of
     writing and reading multi-index columns via a list of tuples. The default in
-    0.11.1 is to write lists of tuples and *not* interpret list of tuples as a 
-    multi-index column.  
+    0.11.1 is to write lists of tuples and *not* interpret list of tuples as a
+    multi-index column.
     Note: The default value will change in 0.12 to make the default *to* write and
     read multi-index columns in the new format. (GH3571_, GH1651_, GH3141_)
   - Add iterator to ``Series.str`` (GH3638_)
@@ -63,14 +63,15 @@ pandas 0.11.1
   - Add modulo operator to Series, DataFrame
   - Add ``date`` method to DatetimeIndex
   - Simplified the API and added a describe method to Categorical
-  - ``melt`` now accepts the optional parameters ``var_name`` and ``value_name`` 
+  - ``melt`` now accepts the optional parameters ``var_name`` and ``value_name``
     to specify custom column names of the returned DataFrame (GH3649_),
     thanks @hoechenberger
   - ``read_html`` no longer performs hard date conversion
   - Plotting functions now raise a ``TypeError`` before trying to plot anything
     if the associated objects have have a dtype of ``object`` (GH1818_,
     GH3572_). This happens before any drawing takes place which elimnates any
     spurious plots from showing up.
+  - Added Faq section on repr display options, to help users customize their setup.
 
 **API Changes**
 
@@ -87,8 +88,8 @@ pandas 0.11.1
     ``timedelta64[ns]`` to ``object/int`` (GH3425_)
   - Do not allow datetimelike/timedeltalike creation except with valid types
     (e.g. cannot pass ``datetime64[ms]``) (GH3423_)
-  - Add ``squeeze`` keyword to ``groupby`` to allow reduction from 
-    DataFrame -> Series if groups are unique. Regression from 0.10.1, 
+  - Add ``squeeze`` keyword to ``groupby`` to allow reduction from
+    DataFrame -> Series if groups are unique. Regression from 0.10.1,
     partial revert on (GH2893_) with (GH3596_)
   - Raise on ``iloc`` when boolean indexing with a label based indexer mask
     e.g. a boolean Series, even with integer labels, will raise. Since ``iloc``
@@ -99,6 +100,8 @@ pandas 0.11.1
   - ``DataFrame.interpolate()`` is now deprecated. Please use
     ``DataFrame.fillna()`` and ``DataFrame.replace()`` instead (GH3582_,
     GH3675_, GH3676_).
+  - Deprecated display.height, display.width is now only a formatting option
+    does not control triggering of summary, similar to < 0.11.0.
 
 **Bug Fixes**
 
@@ -148,11 +151,13 @@ pandas 0.11.1
     is a ``list`` or ``tuple``.
   - Fixed bug where a time-series was being selected in preference to an actual column name
     in a frame (GH3594_)
-  - Fix modulo and integer division on Series,DataFrames to act similary to ``float`` dtypes to return 
+  - Fix modulo and integer division on Series,DataFrames to act similary to ``float`` dtypes to return
     ``np.nan`` or ``np.inf`` as appropriate (GH3590_)
   - Fix incorrect dtype on groupby with ``as_index=False`` (GH3610_)
   - Fix ``read_csv`` to correctly encode identical na_values, e.g. ``na_values=[-999.0,-999]``
     was failing (GH3611_)
+  - Disable HTML output in qtconsole again. (GH3657_)
+  - Reworked the new repr display logic, which users found confusing. (GH3663_)
   - Fix indexing issue in ndim >= 3 with ``iloc`` (GH3617_)
   - Correctly parse date columns with embedded (nan/NaT) into datetime64[ns] dtype in ``read_csv``
     when ``parse_dates`` is specified (GH3062_)

doc/source/faq.rst

@@ -35,8 +35,8 @@ horizontal scrolling, auto-detection of width/height.
 To appropriately address all these environments, the display behavior is controlled
 by several options, which you're encouraged to tweak to suit your setup.
 
-As of 0.11.0, the relavent options are all under the `display` namespace,
-(e.g. display.width, display.height, etc'):
+As of 0.11.1, these are the relavent options, all under the `display` namespace,
+(e.g. display.width,  etc'):
 - notebook_repr_html: if True, IPython frontends with HTML support will display
   dataframes as HTML tables when possible.
 - expand_repr (default True):  when the frame width cannot fit within the screen,
@@ -45,10 +45,10 @@ As of 0.11.0, the relavent options are all under the `display` namespace,
 - max_columns: max dataframe columns to display. a wider frame will trigger
   a summary view, unless `expand_repr` is True and HTML output is disabled.
 - max_rows: max dataframe rows display. a longer frame will trigger a summary view.
-- width: width of display screen in characters. When using a terminal, setting this to None
-  will trigger auto-detection of terminal width.
-- height: height of display screen. When using a terminal, setting this to None
-  will trigger auto-detection of terminal height.
+- width: width of display screen in characters, used to determine the width of lines
+  when expand_repr is active,  Setting this to None will trigger auto-detection of terminal
+  width, this only works for proper terminals, not IPython frontends such as ipnb.
+  width is ignored in IPython notebook, since the browser provides horizontal scrolling.
 
 IPython users can use the IPython startup file to import pandas and set these
 options automatically when starting up.

pandas/core/common.py

@@ -1907,8 +1907,23 @@ def in_qtconsole():
             return True
     except:
         return False
+    return False
+
+def in_ipnb():
+    """
+    check if we're inside an IPython Notebook
+    """
+    try:
+        ip = get_ipython()
+        front_end = (ip.config.get('KernelApp',{}).get('parent_appname',"") or
+                         ip.config.get('IPKernelApp',{}).get('parent_appname',""))
+        if 'notebook' in front_end.lower():
+            return True
+    except:
+        return False
+    return False
 
-def in_ipnb_frontend():
+def in_ipython_frontend():
     """
     check if we're inside an an IPython zmq frontend
     """

pandas/core/config_init.py

@@ -120,13 +120,17 @@
 
 pc_line_width_doc = """
 : int
-    When printing wide DataFrames, this is the width of each line.
+    Deprecated.
 """
 
 pc_line_width_deprecation_warning = """\
 line_width has been deprecated, use display.width instead (currently both are identical)
 """
 
+pc_height_deprecation_warning = """\
+height has been deprecated.
+"""
+
 pc_width_doc = """
 : int
     Width of the display in characters. In case python/IPython is running in
@@ -138,10 +142,7 @@
 
 pc_height_doc = """
 : int
-    Height of the display in lines. In case python/IPython is running in a
-    terminal this can be set to None and pandas will auto-detect the width.
-    Note that the IPython notebook, IPython qtconsole, or IDLE do not run
-    in a terminal, and hence it is not possible to correctly detect the height.
+    Deprecated.
 """
 
 pc_chop_threshold_doc = """
@@ -244,10 +245,15 @@ def mpl_style_cb(key):
                        validator=is_instance_factory([type(None), int]))
     # redirected to width, make defval identical
     cf.register_option('line_width', get_default_val('display.width'), pc_line_width_doc)
+
 cf.deprecate_option('display.line_width',
                     msg=pc_line_width_deprecation_warning,
                     rkey='display.width')
 
+cf.deprecate_option('display.height',
+                    msg=pc_height_deprecation_warning,
+                    rkey='display.height')
+
 tc_sim_interactive_doc = """
 : boolean
     Whether to simulate interactive mode for purposes of testing

pandas/core/format.py

@@ -1702,7 +1702,7 @@ def detect_console_encoding():
 def get_console_size():
     """Return console size as tuple = (width, height).
 
-    May return (None,None) in some cases.
+    Returns (None,None) in non-interactive session.
     """
     display_width = get_option('display.width')
     display_height = get_option('display.height')
@@ -1718,7 +1718,7 @@ def get_console_size():
     # Simple. yeah.
 
     if com.in_interactive_session():
-        if com.in_ipnb_frontend():
+        if com.in_ipython_frontend():
             # sane defaults for interactive non-shell terminal
             # match default for width,height in config_init
             from pandas.core.config import get_default_val

pandas/core/frame.py

@@ -605,57 +605,62 @@ def __nonzero__(self):
 
     def _repr_fits_vertical_(self):
         """
-        Check if full repr fits in vertical boundaries imposed by the display
-        options height and max_rows.  In case of non-interactive session,
-        no boundaries apply.
+        Check length against max_rows.
         """
-        width, height = fmt.get_console_size()
         max_rows = get_option("display.max_rows")
+        return len(self) <= max_rows
 
-        if height is None and max_rows is None:
-            return True
-
-        else:
-            # min of two, where one may be None
-            height = height or max_rows +1
-            max_rows = max_rows or height +1
-            return len(self) <= min(max_rows, height)
-
-    def _repr_fits_horizontal_(self):
+    def _repr_fits_horizontal_(self,ignore_width=False):
         """
         Check if full repr fits in horizontal boundaries imposed by the display
         options width and max_columns. In case off non-interactive session, no
         boundaries apply.
+
+        ignore_width is here so ipnb+HTML output can behave the way
+        users expect. display.max_columns remains in effect.
+        GH3541, GH3573
         """
+
         width, height = fmt.get_console_size()
         max_columns = get_option("display.max_columns")
         nb_columns = len(self.columns)
 
         # exceed max columns
         if ((max_columns and nb_columns > max_columns) or
-            (width and nb_columns > (width // 2))):
+            ((not ignore_width) and width and nb_columns > (width // 2))):
             return False
 
-        if width is None:
-            # no sense finding width of repr if no width set
+        if (ignore_width  # used by repr_html under IPython notebook
+            or not com.in_interactive_session()): # scripts ignore terminal dims
             return True
 
+        if (get_option('display.width') is not None or
+            com.in_ipython_frontend()):
+            # check at least the column row for excessive width
+            max_rows = 1
+        else:
+            max_rows = get_option("display.max_rows")
+
+        # when auto-detecting, so width=None and not in ipython front end
+        # check whether repr fits horizontal by actualy checking
+        # the width of the rendered repr
         buf = StringIO()
 
         # only care about the stuff we'll actually print out
         # and to_string on entire frame may be expensive
         d = self
-        max_rows = get_option("display.max_rows")
-        if not (height is None and max_rows is None):
+
+        if not (max_rows is None): # unlimited rows
             # min of two, where one may be None
-            height = height or max_rows +1
-            max_rows = max_rows or height +1
-            d=d.iloc[:min(max_rows, height,len(d))]
+            d=d.iloc[:min(max_rows,len(d))]
+        else:
+            return True
 
         d.to_string(buf=buf)
         value = buf.getvalue()
         repr_width = max([len(l) for l in value.split('\n')])
-        return repr_width <= width
+
+        return repr_width < width
 
     def __str__(self):
         """
@@ -697,14 +702,11 @@ def __unicode__(self):
         if fits_vertical and fits_horizontal:
             self.to_string(buf=buf)
         else:
-            width, height = fmt.get_console_size()
-            max_rows = get_option("display.max_rows") or height
-            # expand_repr basically takes the extrac columns that don't
-            # fit the width, and creates a new page, which increases
-            # the effective row count. check number of cols agaibst
-            # max rows to catch wrapping. that would exceed max_rows.
-            if (get_option("display.expand_frame_repr") and fits_vertical and
-                len(self.columns) < max_rows):
+            width, _ = fmt.get_console_size()
+            max_rows = get_option("display.max_rows")
+            if (get_option("display.expand_frame_repr")
+                and fits_vertical):
+                # and len(self.columns) < max_rows)
                 self.to_string(buf=buf, line_width=width)
             else:
                 max_info_rows = get_option('display.max_info_rows')
@@ -731,12 +733,22 @@ def _repr_html_(self):
         Return a html representation for a particular DataFrame.
         Mainly for IPython notebook.
         """
+        # ipnb in html repr mode allows scrolling
+        # users strongly prefer to h-scroll a wide HTML table in the browser
+        # then to get a summary view. GH3541, GH3573
+        ipnbh = com.in_ipnb() and get_option('display.notebook_repr_html')
+
+        # qtconsole doesn't report it's line width, and also
+        # behaves badly when outputting an HTML table
+        # that doesn't fit the window, so disable it.
+        if com.in_qtconsole():
+            raise ValueError('Disable HTML output in QtConsole')
 
         if get_option("display.notebook_repr_html"):
             fits_vertical = self._repr_fits_vertical_()
             fits_horizontal = False
             if fits_vertical:
-                fits_horizontal = self._repr_fits_horizontal_()
+                fits_horizontal = self._repr_fits_horizontal_(ignore_width=ipnbh)
 
             if fits_horizontal and fits_vertical:
                 return ('<div style="max-height:1000px;'
@@ -870,7 +882,7 @@ def __contains__(self, key):
 
     # Python 2 division methods
     if not py3compat.PY3:
-        __div__ = _arith_method(operator.div, '__div__', '/', 
+        __div__ = _arith_method(operator.div, '__div__', '/',
                                 default_axis=None, fill_zeros=np.inf)
         __rdiv__ = _arith_method(lambda x, y: y / x, '__rdiv__',
                                  default_axis=None, fill_zeros=np.inf)