[C++, master] No constructor with arguments if struct consists only of fixed-length arrays

[saber-nyan]

For example, here is a part of a schema file:

struct TcpStreamMetadata {
  numOfDataPackets:[uint32:2];
  numOfMessages:[uint32:2];
  numOfBytes:[uint32:2];
}

table TcpStream {
  metadata:TcpStreamMetadata (required);

  srcPort:uint16;
  dstPort:uint16;
  startTimeUsec:uint64;
  endTimeUsec:uint64;
}

I run flatc --cpp -o ./flatbuffers/ --no-prefix --scoped-enums --gen-name-strings --gen-compare --cpp-str-flex-ctor ./flatbuff-schema/redis_tcp_stream_publish.fbs, then the following C++ header is generated from the schema:

FLATBUFFERS_MANUALLY_ALIGNED_STRUCT(4) TcpStreamMetadata FLATBUFFERS_FINAL_CLASS {
 private:
  uint32_t numOfDataPackets_[2];
  uint32_t numOfMessages_[2];
  uint32_t numOfBytes_[2];

 public:
  static FLATBUFFERS_CONSTEXPR const char *GetFullyQualifiedName() {
    return "PackmatePP.FlatBuf.TcpStreamMetadata";
  }
  TcpStreamMetadata() {
    memset(static_cast<void *>(this), 0, sizeof(TcpStreamMetadata));
  }
  const flatbuffers::Array<uint32_t, 2> *numOfDataPackets() const {
    return reinterpret_cast<const flatbuffers::Array<uint32_t, 2> *>(numOfDataPackets_);
  }
  const flatbuffers::Array<uint32_t, 2> *numOfMessages() const {
    return reinterpret_cast<const flatbuffers::Array<uint32_t, 2> *>(numOfMessages_);
  }
  const flatbuffers::Array<uint32_t, 2> *numOfBytes() const {
    return reinterpret_cast<const flatbuffers::Array<uint32_t, 2> *>(numOfBytes_);
  }
};
FLATBUFFERS_STRUCT_END(TcpStreamMetadata, 24);

inline bool operator==(const TcpStreamMetadata &lhs, const TcpStreamMetadata &rhs) {
  return
      (lhs.numOfDataPackets() == rhs.numOfDataPackets()) &&
      (lhs.numOfMessages() == rhs.numOfMessages()) &&
      (lhs.numOfBytes() == rhs.numOfBytes());
}

inline bool operator!=(const TcpStreamMetadata &lhs, const TcpStreamMetadata &rhs) {
    return !(lhs == rhs);
}

How can I create TcpStreamMetadata instance with some data (numOfDataPackets, numOfBytes, ... ), if there is no constructor other than default?

Thanks!

[aardappel]

That's odd indeed :)
It seems it was intended to skip array arguments, not skip the whole constructor:

flatbuffers/src/idl_gen_cpp.cpp

Line 2911 in cb4d0f7

// Generate a constructor that takes all fields as arguments,

@vglavnyy

[vglavnyy]

How can I create TcpStreamMetadata instance with some data (numOfDataPackets, numOfBytes, ... ), if there is no constructor other than default?

It was my decision not to generate constructor with an initializer for arrays (#5313).
Initially, for array_test.fbs/ArrayStruct it was:

ArrayStruct(float _a, const int32_t *_b, int8_t _c, const NestedStruct *_d)

I rejected it: #5313 (comment)

1. What to do if one wants to create struct without initialization of array members (assume a table has 10 scalars, 2 arrays)?
2. I am convinced that if there should be an initializer, then it should only be like std::span.
   Any other variants like std:vector(array), ref to another array, ref to a user-defined container, all of them are not generic. With C++17/20 we can use auto for args but with C++11* don't.
3. For the case of span, a generated struct can has two create constructors:

• =default without any initializers
• ctor(initializers for every field)

We can try to add a generic solution with a light-weight span for code generated with --cpp_std=C++17.
Also, we can add a cast operator from array to span.

@svenk177 might have an opinion about this issue

P.S. As a workaround, it is not complex to write a factory method for TcpStreamMetadata.

[svenk177]

@vglavnyy
I agree there should be another constructor for this.
Can you elaborate what the issue with std::array was?
It should work with C++11 onward and it even allows under-specifying an array, e.g.
std::array<int, 3> arr = { 1, 2 };

[vglavnyy]

There is no problem with std::array except only empty value.

struct fbs_foo {
  arr_field : [byte:50000];
}

fbs_foo:fbs_:foo(const std::array<char, 50000>& arr) { memcpy(arr, field, 50000); }

fbs_foo x({}); // <- emtpy array may be created on the stack and passed to the ctor for copy.

There are many types that can be used as an initialization value:

1. flatbuffers::Array<type, N> (constexpr size()).
2. std::array<type, N> (constexpr size()).
3. arrays with constexpr size() from abseil, boost, or folly libs.
   4 std::base_string_view (usually non-constexpr).
4. ordinal array like char data[N].
5. std::span or gsl::span with or without dynamic extent.
6. std::vector or similar from abseil, boost, folly.
7. pair of contiguous_iterator.
8. std::initializer_list.

The std::span (or gsl::span) is an ideal and well documented proxy object which can cover cases (1-8). Containers like string_view are kind of span, they have data() and size().

There are several ways to pass a variety of initializers into a generated constructor:

1. Restrict to only one type std::array<type, N> or flatbuffers::Array<type, N> with not-solved empty problem.
2. Generate templated ctor with template type Tn for each Array field. The types must meet to a data()+size() concept.
3. Add own flatbuffers::span<T, N> and use it.

I prefer the third way with extremely lightweight flatbuffers::span.
The flatbuffers::Array<type, N> should accept flatbuffers::span as an initializer.
The flatbuffers::span<T, N> should implement std::size_t Extent = std::dynamic_extent like std::span and gsl::span do.
A span may be empty if it has size=dynamic_extent and size=0.
Fixed-size span can be converted into dynamic_extent span.
By default the flatbuffers::span can be constructed from:

• flatbuffers::Vector
• flatbuffers::Array
• C-style raw array like std::span does
• std::array
• std::vector
• std::base_string_vew
• std::base_string

Limitation of generated constructors: there is no way to signal about size() error without std::exception.

An example of contructor for the above mentioned TcpStreamMetadata:

  //struct TcpStreamMetadata {
  //  numOfDataPackets:[uint32:2];
  //  numOfMessages:[uint32:2];
  //  numOfBytes:[uint32:2];
  //}

  TcpStreamMetadata() {
    memset(static_cast<void *>(this), 0, sizeof(TcpStreamMetadata));
  }
  // Every span is a static extent span
  TcpStreamMetadata(flatbuffes::span<uint32_t,2> _numOfDataPackets, /*others*/) {
    memcpy(numOfDataPackets_, _numOfDataPackets.data(), 2);
  }
  
  // Dynamic extent if span created in-place from std::vector or from pair of contiguous_iterator
  TcpStreamMetadata(flatbuffes::span<uint32_t,flatbuffers::dynamic_extent> _numOfDataPackets, /*others*/) {
    FLATBUFFERS_ASSERT(_numOfDataPackets.size() == 2);
    memcpy(numOfDataPackets_, _numOfDataPackets.data(), std::min(_numOfDataPackets.size(), 2u));
  }

Possible problems: legacy compilers with limited support of C++11 or STL.

To avoid a problem with C++11 we can switch the Flatbuffers tests to C++11, almost everything is ready for this. Also, it is possible to test the new feature inside the cpp17/test_cpp17.cpp module.

Unfortunately, now I do not have enough free time to make full implementation but I can prepare a sketch for it. If someone ready to make PR for this I can help.

[aardappel]

I love span and am sad its not in C++ yet.. I am personally using the version from GSL at the moment.

So I'd be tempted to add our own.. but we risk it it not being so small, if it needs to deal with converting from all those types, and have all the usual operators etc.

So maybe keeping it simple and just using 2 args for every array (T* and size_t) would be simpler? Or generating span if the C++20 option has been set? :)

[vglavnyy]

If add simple pair (T*, size) now it will be hard to change this in the future.
I propose to add a simple flatbuffers::span with only one constructor from (T*,size).
Without a dynamic extent and without methods except data() and size().
We can add new ctors and the dynamic extent late.

The std::span or gsl::span can't be used directly if we want to add own classes like Vector or Array. I think it can be solved with CRTP inheritance from std::span.

I'm making the minimal implementation of flatbuffers::span this weekend to exam how it can be used.

[aardappel]

Ok, fair enough, add a simple one. Maybe in stl_emulation.h where we already have some minimal stl replacements, with the idea that later if std::span is available we replace it?

[Deanseo]

Any update on this one so far?

[vglavnyy]

I have partly-implemented code, generated ctors will look like:

 ArrayStruct(
  float _a,
  flatbuffers::span<const int32_t, 15> _b,
  int8_t _c,
  flatbuffers::span<const MyGame::Example::NestedStruct, 2> _d,
  int32_t _e, 
  flatbuffers::span<const int64_t, 2> _f) {...}

template<typename = void> ArrayStruct(
  float _a,
  flatbuffers::span<const int32_t> _b,
  int8_t _c,
  flatbuffers::span<const MyGame::Example::NestedStruct> _d,
  int32_t _e,
  flatbuffers::span<const int64_t> _f) {...}

I stuck with MSV2010 and wasted time with a workaround for it.
Unfortunately, now I don't have enough time to finish this work.
If the problem is not resolved until Jul, I will return to it.
Can you prepare your own PR for this issue (with or without span)?
Prototype with span.