Skip to content

HDDS-12203. Initialize block length before skip #7809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
jojochuang merged 1 commit into from
Feb 4, 2025
Merged

HDDS-12203. Initialize block length before skip #7809

jojochuang merged 1 commit into from
Feb 4, 2025

Conversation

@oneonestar
Copy link
Contributor

@oneonestar oneonestar commented Feb 4, 2025

What changes were proposed in this pull request?

HDDS-12203. Initialize block length before skip

Please describe your PR in detail:

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12203

How was this patch tested?

Integration Tests

@oneonestar oneonestar mentioned this pull request Feb 4, 2025
Merged
jojochuang
jojochuang approved these changes Feb 4, 2025
View reviewed changes
Copy link
Contributor

@jojochuang jojochuang left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to ask to add test code to cover it, but realized that there are already existing tests:

ozone/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestHSync.java

Line 816 in 93dab91

final long skipped = in.skip(length);

ozone/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/read/TestKeyInputStream.java

Line 349 in 477c8de

long skipped = keyInputStream.skip(70);

Note: OzoneFSInputStream.skip() doesn't fail in these existing tests because it doesn't implement skip(). Once #7801 is implemented it will start to fail.

@jojochuang jojochuang merged commit 7efa081 into apache:master Feb 4, 2025
1 check passed
@jojochuang
Copy link
Contributor

jojochuang commented Feb 4, 2025

Merged. Thanks @oneonestar welcome to the Ozone community!

@adoroszlai
Copy link
Contributor

adoroszlai commented Feb 4, 2025

Thanks @oneonestar for the patch, @jojochuang for the review. Please trigger CI by "Approve and run" before merge. (Unfortunately the labeler workflow runs without approval, adding a ✅ to the PR, which can be easily confused with the full CI workflow.)

@oneonestar oneonestar deleted the star/fix_skip branch February 5, 2025 01:41
@oneonestar
Copy link
Contributor Author

oneonestar commented Feb 5, 2025

@jojochuang @adoroszlai Thanks for the quick review =)

errose28 added a commit to errose28/ozone that referenced this pull request Feb 5, 2025
@errose28
Merge branch 'master' into HDDS-10239-container-reconciliation
3797c28 
* master: (168 commits)
  HDDS-12112. Fix interval used for Chunk Read/Write Dashboard (apache#7724)
  HDDS-12212. Fix grammar in decommissioning and observability documentation (apache#7815)
  HDDS-12195. Implement skip() in OzoneFSInputStream (apache#7801)
  HDDS-12200. Fix grammar in OM HA, EC and Snapshot doc (apache#7806)
  HDDS-12202. OpsCreate and OpsAppend metrics not incremented (apache#7811)
  HDDS-12203. Initialize block length before skip (apache#7809)
  HDDS-12183. Reuse cluster across safe test classes (apache#7793)
  HDDS-11714. resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster. (apache#7665)
  HDDS-12186. (addendum) Avoid array allocation for table iterator (apache#7799)
  HDDS-12186. Avoid array allocation for table iterator. (apache#7797)
  HDDS-11508. Decouple delete batch limits from Ratis request size for DirectoryDeletingService. (apache#7365)
  HDDS-12073. Don't show Source Bucket and Volume if null in DU metadata (apache#7760)
  HDDS-12142. Save logs from build check (apache#7782)
  HDDS-12163. Reduce number of individual getCapacity/getAvailable/getUsedSpace calls (apache#7790)
  HDDS-12176. Trivial dependency cleanup.(apache#7787)
  HDDS-12181. Bump jline to 3.29.0 (apache#7789)
  HDDS-12165. Refactor VolumeInfoMetrics to use getCurrentUsage (apache#7784)
  HDDS-12085. Add manual refresh button for DU page (apache#7780)
  HDDS-12132. Parameterize testUpdateTransactionInfoTable for SCM (apache#7768)
  HDDS-11277. Remove dependency on hadoop-hdfs in Ozone client (apache#7781)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java
hadoop-ozone/dist/src/main/smoketest/admincli/container.robot
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/ClosedContainerReplicator.java
errose28 added a commit to errose28/ozone that referenced this pull request Feb 6, 2025
@errose28
Merge branch 'master' into HDDS-10239-container-reconciliation
2cfd22a 
* master: (168 commits)
  HDDS-12112. Fix interval used for Chunk Read/Write Dashboard (apache#7724)
  HDDS-12212. Fix grammar in decommissioning and observability documentation (apache#7815)
  HDDS-12195. Implement skip() in OzoneFSInputStream (apache#7801)
  HDDS-12200. Fix grammar in OM HA, EC and Snapshot doc (apache#7806)
  HDDS-12202. OpsCreate and OpsAppend metrics not incremented (apache#7811)
  HDDS-12203. Initialize block length before skip (apache#7809)
  HDDS-12183. Reuse cluster across safe test classes (apache#7793)
  HDDS-11714. resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster. (apache#7665)
  HDDS-12186. (addendum) Avoid array allocation for table iterator (apache#7799)
  HDDS-12186. Avoid array allocation for table iterator. (apache#7797)
  HDDS-11508. Decouple delete batch limits from Ratis request size for DirectoryDeletingService. (apache#7365)
  HDDS-12073. Don't show Source Bucket and Volume if null in DU metadata (apache#7760)
  HDDS-12142. Save logs from build check (apache#7782)
  HDDS-12163. Reduce number of individual getCapacity/getAvailable/getUsedSpace calls (apache#7790)
  HDDS-12176. Trivial dependency cleanup.(apache#7787)
  HDDS-12181. Bump jline to 3.29.0 (apache#7789)
  HDDS-12165. Refactor VolumeInfoMetrics to use getCurrentUsage (apache#7784)
  HDDS-12085. Add manual refresh button for DU page (apache#7780)
  HDDS-12132. Parameterize testUpdateTransactionInfoTable for SCM (apache#7768)
  HDDS-11277. Remove dependency on hadoop-hdfs in Ozone client (apache#7781)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java
hadoop-ozone/dist/src/main/smoketest/admincli/container.robot
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/ClosedContainerReplicator.java
errose28 added a commit to errose28/ozone that referenced this pull request Feb 7, 2025
@errose28
Merge branch 'HDDS-10239-container-reconciliation' into HDDS-12079-tr…
dceb3ef 
…ee-improvements

* HDDS-10239-container-reconciliation: (168 commits)
  HDDS-12112. Fix interval used for Chunk Read/Write Dashboard (apache#7724)
  HDDS-12212. Fix grammar in decommissioning and observability documentation (apache#7815)
  HDDS-12195. Implement skip() in OzoneFSInputStream (apache#7801)
  HDDS-12200. Fix grammar in OM HA, EC and Snapshot doc (apache#7806)
  HDDS-12202. OpsCreate and OpsAppend metrics not incremented (apache#7811)
  HDDS-12203. Initialize block length before skip (apache#7809)
  HDDS-12183. Reuse cluster across safe test classes (apache#7793)
  HDDS-11714. resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster. (apache#7665)
  HDDS-12186. (addendum) Avoid array allocation for table iterator (apache#7799)
  HDDS-12186. Avoid array allocation for table iterator. (apache#7797)
  HDDS-11508. Decouple delete batch limits from Ratis request size for DirectoryDeletingService. (apache#7365)
  HDDS-12073. Don't show Source Bucket and Volume if null in DU metadata (apache#7760)
  HDDS-12142. Save logs from build check (apache#7782)
  HDDS-12163. Reduce number of individual getCapacity/getAvailable/getUsedSpace calls (apache#7790)
  HDDS-12176. Trivial dependency cleanup.(apache#7787)
  HDDS-12181. Bump jline to 3.29.0 (apache#7789)
  HDDS-12165. Refactor VolumeInfoMetrics to use getCurrentUsage (apache#7784)
  HDDS-12085. Add manual refresh button for DU page (apache#7780)
  HDDS-12132. Parameterize testUpdateTransactionInfoTable for SCM (apache#7768)
  HDDS-11277. Remove dependency on hadoop-hdfs in Ozone client (apache#7781)
  ...
nandakumar131 pushed a commit to nandakumar131/ozone that referenced this pull request Feb 10, 2025
@oneonestar @nandakumar131
HDDS-12203. Initialize block length before skip (apache#7809)
a94f111
swamirishi pushed a commit to swamirishi/ozone that referenced this pull request Dec 3, 2025
@oneonestar @rohit-kb
CDPD-78731: HDDS-12203. Initialize block length before skip (apache#7809
2d93523 
)

Change-Id: If349747bcdae6609f3bcfa854234ddf1807f11a6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jojochuang jojochuang jojochuang approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@oneonestar @jojochuang @adoroszlai