Skip to content

fix:remove goroutine#70

Merged
j3ttt merged 3 commits intoantgroup:mainfrom
j3ttt:fix/remove-goroutine
Sep 2, 2025
Merged

fix:remove goroutine#70
j3ttt merged 3 commits intoantgroup:mainfrom
j3ttt:fix/remove-goroutine

Conversation

@j3ttt
Copy link
Collaborator

@j3ttt j3ttt commented Sep 2, 2025

Thank you for your contribution to CloudRec!

What About:

  • Server (java)
  • Collector (go)
  • Rule (opa)

Description:

fix: remove goroutine concurrency as it increased complexity without significantly improving collection speed

Summary by Sourcery

Remove unnecessary goroutine concurrency from multiple collector modules and simplify resource detail loops, add dedicated Cognito Identity Pool collector, and upgrade AliCloud SWAS client to the new swas-open-20200601 SDK with a helper initializer.

New Features:

  • Added AWS Cognito Identity Pool collector
  • Updated AliCloud SWAS client to the swas-open-20200601 SDK and introduced createSWASClient

Enhancements:

  • Removed goroutine-based parallelism and sync.WaitGroup usage across collector modules in favor of straightforward sequential loops
  • Simplified and inlined resource detail retrieval logic by eliminating describe*Detail helper functions and channels
  • Cleaned up imports and reordered AWS and AliCloud service client initializations in services.go

@gemini-code-assist
Copy link
Contributor

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

@sourcery-ai
Copy link

sourcery-ai bot commented Sep 2, 2025

Reviewer's Guide

This PR removes goroutine-based worker pools and sync primitives from collector modules in favor of simple sequential loops, inlines and consolidates detail-fetching logic, updates AWS and AliCloud service client imports and initialization, and adds support for Cognito Identity Pools.

Sequence diagram for detail aggregation after goroutine removal

sequenceDiagram
    participant Collector
    participant AWSServiceClient
    participant ResourceDetail
    Collector->>AWSServiceClient: List resources (e.g., ListUsers)
    AWSServiceClient-->>Collector: Return resource list
    loop For each resource
        Collector->>AWSServiceClient: Fetch details (e.g., list policies, tags)
        AWSServiceClient-->>Collector: Return details
        Collector->>ResourceDetail: Aggregate details
    end
    Collector->>ResourceDetail: Send aggregated detail to result channel
Loading

Class diagram for updated Cognito Identity Pool support

classDiagram
    class IdentityPoolDetail {
        +IdentityPool: IdentityPoolShortDescription
        +Tags: map[string]string
    }
    class CognitoIdentityPoolShortDescription {
        <<external>>
    }
    IdentityPoolDetail --> CognitoIdentityPoolShortDescription: contains
Loading

Class diagram for refactored detail aggregation types

classDiagram
    class UserPoolDetail {
        +UserPool: UserPoolDescriptionType
        +UserPoolClients: []UserPoolClientDescription
        +Users: []UserType
        +Tags: map[string]string
    }
    class RoleDetail {
        +Role: Role
        +AttachedPolicies: []AttachedPolicy
        +InlinePolicies: []string
        +Tags: []Tag
    }
    class UserDetail {
        +User: User
        +AttachedPolicies: []AttachedPolicy
        +InlinePolicies: []string
        +MFADevices: []MFADevice
        +AccessKeys: []AccessKeyMetadata
        +LoginProfile: GetLoginProfileOutput
        +Tags: []Tag
    }
    class PolicyDetail {
        +Policy: Policy
        +Version: PolicyVersion
        +Tags: []Tag
    }
    class TableDetail {
        +Table: TableDescription
        +ContinuousBackups: ContinuousBackupsDescription
    }
    class LogGroupDetail {
        +LogGroup: LogGroup
        +MetricFilters: []MetricFilter
        +ResourcePolicies: []ResourcePolicy
        +Tags: map[string]string
    }
    class AlarmDetail {
        +MetricAlarm: MetricAlarm
        +Tags: []Tag
    }
    class FleetDetail {
        +Fleet: Fleet
        +Tags: []Tag
    }
    class RecorderDetail {
        +Recorder: ConfigurationRecorder
        +Status: ConfigurationRecorderStatus
        +DeliveryChannel: DeliveryChannel
    }
    class SecretDetail {
        +Secret: SecretListEntry
        +Policy: map[string]interface
    }
    class SQSQueueDetail {
        +Url: string
        +Attributes: map[string]string
        +Policy: map[string]interface
        +Tags: map[string]string
    }
    class SNSTopicDetail {
        +Topic: Topic
        +Attributes: map[string]string
        +Policy: map[string]interface
        +Subscriptions: []Subscription
        +Tags: []Tag
    }
    class DetectorDetail {
        +Detector: Detector
        +Administrator: Administrator
        +Tags: map[string]string
    }
    class APIV2Detail {
        +API: Api
        +Stages: []Stage
        +Authorizers: []Authorizer
        +Tags: map[string]string
    }
    class ClusterDetail {
        +Cluster: Cluster
        +Services: []Service
        +Tasks: []Task
    }
    class FunctionDetail {
        +Function: FunctionConfiguration
        +Policy: map[string]interface
        +UrlConfigs: []FunctionUrlConfig
        +Tags: map[string]string
    }
    class CertificateDetail {
        +Certificate: CertificateDetail
        +Tags: []Tag
    }
    class FirewallDetail {
        +Firewall: Firewall
        +FirewallStatus: FirewallStatus
        +UpdateToken: string
    }
    class RuleGroupDetail {
        +RuleGroupResponse: RuleGroupResponse
        +UpdateToken: string
        +RuleGroup: RuleGroup
    }
    class DomainDetail {
        +DomainStatus: DomainStatus
    }
    class InstanceDetail {
        +Instance: InstanceVO
        +InstanceBaseInfo: InstanceBaseInfo
    }
    class BastionhostDetail {
        +Instance: Instances
        +InstanceAttribute: InstanceAttribute
    }
    class CoverageDetail {
        +Coverage: CoveredResource
    }
    class ClassificationJobDetail {
        +Job: JobSummary
        +Tags: map[string]string
    }
    class FindingDetail {
        +Finding: Finding
        +Tags: map[string]string
    }
Loading

File-Level Changes

Change Details Files
Eliminate goroutine-based concurrency and sync primitives across collectors
  • Removed maxWorkers constants, sync.WaitGroup and channel-based task queues
  • Replaced parallel worker goroutines with direct for-loops sending results
  • Cleaned up unused sync imports
collector/aws/collector/**/*.go
collector/alicloud/collector/**/*.go
Update service client imports and initialization for AWS and AliCloud
  • Reordered and added AWS SDK imports in AWS services.go
  • Introduced Alibaba SWAS v3 import and createSWASClient in AliCloud services.go
  • Updated alicloud/go.mod to reference swas-open-20200601
collector/aws/collector/services.go
collector/alicloud/collector/services.go
collector/alicloud/go.mod
Add Cognito Identity Pool support
  • Created new identitypool.go with resource definition and detail fetching
  • Added listIdentityPools and listIdentityPoolTags helpers
collector/aws/collector/cognito/identitypool.go

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@jietian-sts jietian-sts self-requested a review September 2, 2025 03:26
@j3ttt j3ttt merged commit a56b461 into antgroup:main Sep 2, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants