[C++] Implement cumulative product, max, and min compute functions

[asfimport]

Other libraries/languages (pandas, R, numpy) have cumprod, cummax and cummin functions that useful to add to Arrow, similar to the now existing cumulative sum function.

Reporter: Jabari Booker / @JabariBooker

Related issues:

• [C++] Implement cumulative sum compute function (is related to)

PRs and other links:

• GitHub Pull Request #13412

_{Note: This issue was originally created as ARROW-16865. Please see the migration documentation for further details.}

[asfimport]

Apache Arrow JIRA Bot:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

[frosk1]

Hey, is this coming?

[ianmcook]

@frosk1 As far as I know there is no work underway currently on cumulative vector functions (beyond the cumulative sum vector function which was implemented in #12460). If you're interested to take a stab at implementing one of these in a PR, following the example of the cumulative sum implementation, we would be happy to review.

[js8544]

Hi @frosk1 are you working on this? I plan to implement cumprod, cummax and cummin if no one else is working on it.

[frosk1]

Hey @js8544 no unfortunately I do not have time atm. Would be great if you can work on that one.

[js8544]

take

[js8544]

I noticed that the functions cumprod, cummin and cummax would require the same option as CumulativeSumOptions (min and max won't need to check_overflow but I plan to remove it in #35790).

I want to avoid having four different option types with identical structure, so I intend to refactor CumulativeSumOptions to CumulativeOptions and reuse it for all cumulative functions.

The current definition is

/// \brief Options for cumulative sum function
class ARROW_EXPORT CumulativeSumOptions : public FunctionOptions {
 public:
  explicit CumulativeSumOptions(double start = 0, bool skip_nulls = false,
                                bool check_overflow = false);
  explicit CumulativeSumOptions(std::shared_ptr<Scalar> start, bool skip_nulls = false,
                                bool check_overflow = false);
  static constexpr char const kTypeName[] = "CumulativeSumOptions";
  static CumulativeSumOptions Defaults() { return CumulativeSumOptions(); }

  /// Optional starting value for cumulative operation computation
  std::shared_ptr<Scalar> start;

  /// If true, nulls in the input are ignored and produce a corresponding null output.
  /// When false, the first null encountered is propagated through the remaining output.
  bool skip_nulls = false;

  /// When true, returns an Invalid Status when overflow is detected
  bool check_overflow = false;
};

I plan to do this:

/// \brief Options for cumulative functions
class ARROW_EXPORT CumulativeOptions : public FunctionOptions {
  // ... (same as above)
};

using CumulativeSumOptions = CumulativeOptions; // for backward compatibility

I checked that with this change all compute tests still passed, both in cpp and python. But I want to make sure there are no better ways to solve this. Do you consider this as an acceptable approach? cc @pitrou @westonpace

[pitrou]

@js8544 This seems like a good approach to me.

[westonpace]

I agree with this approach. We have done something very similar with aggregate functions and ScalarAggregateOptions:

arrow/cpp/src/arrow/compute/api_aggregate.h

Lines 45 to 59 in 8b2ab4d

	/// \brief Control general scalar aggregate kernel behavior
	///
	/// By default, null values are ignored (skip_nulls = true).
	class ARROW_EXPORT ScalarAggregateOptions : public FunctionOptions {
	public:
	explicit ScalarAggregateOptions(bool skip_nulls = true, uint32_t min_count = 1);
	static constexpr char const kTypeName[] = "ScalarAggregateOptions";
	static ScalarAggregateOptions Defaults() { return ScalarAggregateOptions{}; }
	/// If true (the default), null values are ignored. Otherwise, if any value is null,
	/// emit null.
	bool skip_nulls;
	/// If less than this many non-null values are observed, emit null.
	uint32_t min_count;
	};

[westonpace]

CC @icexelloss / @rtpsw / @ildipo may be interested in these functions when available. They are classic window functions and so could be useful in the work that is being done to add window function support to Acero.

[js8544]

CC @icexelloss / @rtpsw / @ildipo may be interested in these functions when available. They are classic window functions and so could be useful in the work that is being done to add window function support to Acero.

Regarding window functions. I also implemented a pairwise_diff function (#35787) you guys may also want to take a look at. pairwise functions would corresponds to LAG/LEAD window functions in SQL.

I also plan to implement a series of rolling_* functions that computes over a fixed length sliding window soon, which corresponds to the window option ROWS BETWEEN n PRECEDING AND CURRENT ROW in sql.