Skip to content

Implemented stub partylookup, approval, ordering, splitting, formatting and mergeBack worker#10

Merged
fynmanoj merged 1 commit into
openMF:masterfrom
danishjamal104:stub-worker
Aug 3, 2022
Merged

Implemented stub partylookup, approval, ordering, splitting, formatting and mergeBack worker#10
fynmanoj merged 1 commit into
openMF:masterfrom
danishjamal104:stub-worker

Conversation

@danishjamal104
Copy link
Copy Markdown
Contributor

No description provided.

@danishjamal104
Copy link
Copy Markdown
Contributor Author

@avikganguly01 and @fynmanoj please review and merge. Thanks :)

MappingIterator<Transaction> readValues = csvMapper.readerWithSchemaFor(Transaction.class).with(schema).readValues(csvFile);

while (readValues.hasNext()) {
/*while (readValues.hasNext()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove these commented lines

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will be removed in upcoming PRs, since this PR scope is to only add empty Worker and disabled Kafka, so commented out related code in the entire code base.

fileWriter.close();

logger.info(csvData);
logger.info(""+file.length());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logger.info("{}", file.length()); this is the best way to use logger statements, it removes the overhead of string concatenation.

Copy link
Copy Markdown
Contributor Author

@danishjamal104 danishjamal104 Aug 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is just for testing purposes and will be removed in upcoming PRs.

file.setWritable(true);
file.setReadable(true);

String csvData = exchange.getIn().getBody(String.class);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the max size of the file? I don't think it is the right approach if the file size can be very large.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not focusing on this issue rit now.

Copy link
Copy Markdown
Contributor Author

@danishjamal104 danishjamal104 Aug 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be considered in phase 2 or 3.

MappingIterator<Transaction> readValues = csvMapper.readerWithSchemaFor(Transaction.class).with(schema).readValues(csvFile);

while (readValues.hasNext()) {
/*while (readValues.hasNext()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented lines

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will be removed in upcoming PRs, since this PR scope is to only add empty Worker and disabled Kafka, so commented out related code in the entire code base.

client.containerName(bucketName).blobName(fileName).buildClient().upload(Files.newInputStream(file.toPath()), file.length());
return fileName;
} catch (IOException e) {
logger.error("Error uploading file to Azure", e);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this as per design to suppress the exception here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, If staging fail is the incident. We cant move ahead without this step.

@danishjamal104 danishjamal104 requested a review from fynmanoj August 3, 2022 10:08
@fynmanoj fynmanoj merged commit 13ebfbc into openMF:master Aug 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants