Skip to content

Conversation

@zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Apr 25, 2024

Proposed changes

Issue Number: close #xxx

when there are multiple transactions try to create partitions in one table. they may meet:

2024-04-25 17:54:34,025 WARN (thrift-server-pool-3|319) [FrontendServiceImpl.loadTxnCommit():1559] catch unknown result.
java.util.ConcurrentModificationException: null
	at java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1784) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
	at org.apache.doris.cloud.transaction.CloudGlobalTransactionMgr.getBaseTabletsFromTables(CloudGlobalTransactionMgr.java:393) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.cloud.transaction.CloudGlobalTransactionMgr.commitTransaction(CloudGlobalTransactionMgr.java:422) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.cloud.transaction.CloudGlobalTransactionMgr.commitTransaction(CloudGlobalTransactionMgr.java:323) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.cloud.transaction.CloudGlobalTransactionMgr.commitAndPublishTransaction(CloudGlobalTransactionMgr.java:749) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.service.FrontendServiceImpl.loadTxnCommitImpl(FrontendServiceImpl.java:1603) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.service.FrontendServiceImpl.loadTxnCommit(FrontendServiceImpl.java:1543) ~[doris-fe.jar:1.2-SNAPSHOT]
	at jdk.internal.reflect.GeneratedMethodAccessor111.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
	at org.apache.doris.service.FeServer.lambda$start$0(FeServer.java:60) ~[doris-fe.jar:1.2-SNAPSHOT]
	at jdk.proxy2.$Proxy35.loadTxnCommit(Unknown Source) ~[?:?]
	at org.apache.doris.thrift.FrontendService$Processor$loadTxnCommit.getResult(FrontendService.java:3982) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
	at org.apache.doris.thrift.FrontendService$Processor$loadTxnCommit.getResult(FrontendService.java:3962) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) ~[libthrift-0.16.0.jar:0.16.0]
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) ~[libthrift-0.16.0.jar:0.16.0]
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250) ~[libthrift-0.16.0.jar:0.16.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:840) ~[?:?]

this should merge after run cloud p1 many times

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

@swjtu-zhanglei
Copy link
Contributor

"craeting" does it means "creating" ?

Copy link
Contributor

@swjtu-zhanglei swjtu-zhanglei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@zclllyybb zclllyybb changed the title [Fix](cloud) Fix concurrency bugs on craeting auto partition [Fix](cloud) Fix concurrency bugs on creating auto partition Apr 26, 2024
@zclllyybb
Copy link
Contributor Author

run cloud_p1 10

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 26, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@zhangstar333 zhangstar333 merged commit d6a8e24 into apache:master Apr 26, 2024
@zclllyybb zclllyybb deleted the auto_cloud branch April 27, 2024 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants