Skip to content

Conversation

@hangc0276
Copy link
Contributor

…cksDB version

Fixes #xyz

Master Issue: #xyz

PIP: #xyz

Motivation

Ugrade the RocksDB's version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label May 6, 2023
@hangc0276 hangc0276 self-assigned this May 6, 2023
@hangc0276 hangc0276 changed the title Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version [Branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version May 6, 2023
@hangc0276 hangc0276 changed the title [Branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version [fix] [broker] [Branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version May 6, 2023
@Technoboy- Technoboy- merged commit f88a49e into apache:branch-2.10 May 7, 2023
@dave2wave
Copy link
Member

dave2wave commented May 9, 2023

Since the merge of this some of our Pulsar 2.10 integration tests are failing these include errors in Bookkeeper pods like:

2023-05-09T15:05:19,859+0000 [BookieReadThreadPool-OrderedExecutor-2-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught 
java.lang.NoSuchMethodError: 'org.rocksdb.ReadOptions org.rocksdb.ReadOptions.setIterateUpperBound(org.rocksdb.Slice)'
	at org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.getFloor(KeyValueStorageRocksDB.java:259) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex.getLastEntryInLedgerInternal(EntryLocationIndex.java:118) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex.getLastEntryInLedger(EntryLocationIndex.java:111) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.getLastEntry(SingleDirectoryDbLedgerStorage.java:575) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.getEntry(SingleDirectoryDbLedgerStorage.java:405) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.getLastAddConfirmed(SingleDirectoryDbLedgerStorage.java:771) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.getLastAddConfirmed(DbLedgerStorage.java:205) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.LedgerDescriptorImpl.getLastAddConfirmed(LedgerDescriptorImpl.java:165) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.bookie.Bookie.readLastAddConfirmed(Bookie.java:1527) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.proto.ReadEntryProcessorV3.readEntry(ReadEntryProcessorV3.java:187) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.proto.ReadEntryProcessorV3.readEntry(ReadEntryProcessorV3.java:157) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.proto.ReadEntryProcessorV3.getReadResponse(ReadEntryProcessorV3.java:220) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.proto.ReadEntryProcessorV3.executeOp(ReadEntryProcessorV3.java:273) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.proto.ReadEntryProcessorV3.safeRun(ReadEntryProcessorV3.java:269) ~[org.apache.bookkeeper-bookkeeper-server-4.14.7.jar:4.14.7]
	at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) ~[org.apache.bookkeeper-bookkeeper-common-4.14.7.jar:4.14.7]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.87.Final.jar:4.1.87.Final]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]

hangc0276 added a commit to hangc0276/pulsar that referenced this pull request May 10, 2023
… sync with BookKeeper's RocksDB version (apache#20243)"

This reverts commit f88a49e.
@hangc0276
Copy link
Contributor Author

Hi @dave2wave, I have pushed one PR #20287 to revert this change. The Pulsar RocksDB version upgrade should be done after the BookKeeper side upgradation. Refer to #14962 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants