Cryptic SDSS table bug

[emirkmo]

Reported on Slack by @rhandberg:

ERROR - Could not download catalog
Traceback (most recent call last):
  File "/usr/users/kasoc/flows/flows/run_catalogs.py", line 47, in main
    download_catalog(target)  # @TODO: refactor to Tendrils
  File "/usr/users/kasoc/flows/flows/flows/catalogs.py", line 592, in download_catalog
    tab = query_all(coo_centre, radius=radius, dist_cutoff=dist_cutoff)
  File "/usr/users/kasoc/flows/flows/flows/catalogs.py", line 492, in query_all
    AT_sdss, sdss = query_sdss(coo_centre, radius=radius)
  File "/usr/users/kasoc/flows/flows/flows/catalogs.py", line 301, in query_sdss
    AT_sdss = SDSS.query_region(coo_centre, photoobj_fields=['type', 'clean', 'ra', 'dec', 'psfMag_u'], data_release=dr,
  File "/usr/users/kasoc/python-virtualenv/flows-nerdrage-py3.10/lib/python3.10/site-packages/astroquery/utils/class_or_instance.py", line 25, in f
    return self.fn(obj, *args, **kwds)
  File "/usr/users/kasoc/python-virtualenv/flows-nerdrage-py3.10/lib/python3.10/site-packages/astroquery/utils/process_asyncs.py", line 29, in newmethod
    result = self._parse_result(response, verbose=verbose)
  File "/usr/users/kasoc/python-virtualenv/flows-nerdrage-py3.10/lib/python3.10/site-packages/astroquery/sdss/core.py", line 874, in _parse_result
    arr = np.atleast_1d(np.genfromtxt(io.BytesIO(response.content),
  File "/usr/users/kasoc/python-virtualenv/flows-nerdrage-py3.10/lib/python3.10/site-packages/numpy/lib/npyio.py", line 2291, in genfromtxt
    raise ValueError(errmsg)
ValueError: Some errors were detected !
    Line #62 (got 4 columns instead of 1)
    Line #124 (got 2 columns instead of 1)
    Line #125 (got 2 columns instead of 1)
    Line #131 (got 2 columns instead of 1)
    Line #132 (got 4 columns instead of 1)
    Line #133 (got 2 columns instead of 1)
    Line #134 (got 2 columns instead of 1)
    Line #137 (got 2 columns instead of 1)

This is caused by a combination of factors. We had pinned to old astroquery because of:
astropy/astroquery#2659

But now astropy/astroquery#2654 showed that SDSS queries are broken due to python web client dropping headers on redirects. However this can be worked around with fixing astroquery to a version before the PR that resolved 2659 (from September 2022, actually v0.4.6, the last stable version, still works), but a small workaround is needed:

#SDSS.MAX_CROSSID_RADIUS = radius + 1 * u.arcmin
    sdss.conf.skyserver_baseurl = sdss.conf.skyserver_baseurl.replace("http://","https://")
    AT_sdss = sdss.SDSS.query_region(coo_centre, photoobj_fields=['type', 'clean', 'ra', 'dec', 'psfMag_u'], data_release=dr,
                                timeout=600, radius=radius)

I am making a PR with this fix which makes all tests, including catalog xmatch, tns, and ztf queries pass.