Skip to content

[feat] Persist extraction result in job resultsย #4

@KyungminPark-steck

Description

@KyungminPark-steck

๐Ÿ“– ๋ฐฐ๊ฒฝ

PR #3์—์„œ worker๊ฐ€ Instagram caption ๊ธฐ๋ฐ˜์œผ๋กœ extraction_result๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ๋Š” schema, domain model, HF extraction client, worker extraction interface๊ฐ€ ์ถ”๊ฐ€๋˜์—ˆ๋‹ค.

ํ•˜์ง€๋งŒ ํ˜„์žฌ JobRepository.upsert_job_result()๋Š” extraction_result ๊ฐ’์„ ์ธ์ž๋กœ ๋ฐ›๊ธฐ๋งŒ ํ•˜๊ณ  ์‹ค์ œ DB์—๋Š” ์ €์žฅํ•˜์ง€ ์•Š๋Š”๋‹ค. ๋”ฐ๋ผ์„œ worker๊ฐ€ ์ถ”์ถœํ•œ ๊ฒฐ๊ณผ๊ฐ€ /jobs/{jobId}/result ์‘๋‹ต๊นŒ์ง€ ์•ˆ์ •์ ์œผ๋กœ ์ด์–ด์ง€๋ ค๋ฉด job_results ํ…Œ์ด๋ธ” ์ €์žฅ/์กฐํšŒ ํ๋ฆ„์— extraction_result๋ฅผ ์—ฐ๊ฒฐํ•ด์•ผ ํ•œ๋‹ค.

๋˜ํ•œ DB schema ๋ณ€๊ฒฝ์ด ํ•„์š”ํ•˜๋ฏ€๋กœ ํŒ€ ์š”์ฒญ์— ๋”ฐ๋ผ migration ์ฝ”๋“œ๋„ ํ•จ๊ป˜ ์ถ”๊ฐ€ํ•ด์•ผ ํ•œ๋‹ค.

๐ŸŽฏ ๋ชฉํ‘œ

worker์—์„œ ์ƒ์„ฑํ•œ extraction_result๋ฅผ job_results DB ๋ ˆ์ฝ”๋“œ์— ์ €์žฅํ•˜๊ณ , /jobs/{jobId}/result API ์‘๋‹ต์—์„œ ๋‹ค์‹œ ์กฐํšŒํ•ด ๋ฐ˜ํ™˜ํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•œ๋‹ค.

๐Ÿ› ๏ธ ๊ตฌํ˜„ ๋‚ด์šฉ

  • job_results ํ…Œ์ด๋ธ”์— extraction_result JSONB ์ปฌ๋Ÿผ์„ ์ถ”๊ฐ€ํ•˜๋Š” migration ์ฝ”๋“œ ์ž‘์„ฑ
  • JobRepository.upsert_job_result()์—์„œ extraction_result๋ฅผ DB์— ์ €์žฅํ•˜๋„๋ก ์ˆ˜์ •
  • JobRepository.get_job_result()์—์„œ extraction_result๋ฅผ ์กฐํšŒํ•ด JobResultRecord์— ๋งคํ•‘ํ•˜๋„๋ก ์ˆ˜์ •
  • /jobs/{jobId}/result ์‘๋‹ต์— ์ €์žฅ๋œ extraction_result๊ฐ€ ํฌํ•จ๋˜๋Š”์ง€ ํ™•์ธ
  • ์ €์žฅ/์กฐํšŒ/API ์‘๋‹ต ํ๋ฆ„์„ ๊ฒ€์ฆํ•˜๋Š” ํ…Œ์ŠคํŠธ ์ถ”๊ฐ€
  • ๊ฐ€๋Šฅํ•˜๋ฉด ๋กœ์ปฌ PostgreSQL/Redis ๊ธฐ๋ฐ˜ worker ํ๋ฆ„์—์„œ ์‹ค์ œ ์ €์žฅ ์—ฌ๋ถ€ ํ™•์ธ

โš ๏ธ ๊ณ ๋ ค์‚ฌํ•ญ

  • DB schema ๋ณ€๊ฒฝ์ด ํฌํ•จ๋˜๋ฏ€๋กœ migration ์ฝ”๋“œ๊ฐ€ ๋ฐ˜๋“œ์‹œ ํฌํ•จ๋˜์–ด์•ผ ํ•œ๋‹ค.
  • ๊ธฐ์กด job_results ๋ฐ์ดํ„ฐ๊ฐ€ ์žˆ์–ด๋„ ์•ˆ์ „ํ•˜๊ฒŒ ์ ์šฉ๋  ์ˆ˜ ์žˆ๋„๋ก nullable JSONB ์ปฌ๋Ÿผ์œผ๋กœ ์ถ”๊ฐ€ํ•œ๋‹ค.
  • extraction_result๋Š” LLM ์ถ”์ถœ ์‹คํŒจ ์‹œ null์ผ ์ˆ˜ ์žˆ์–ด์•ผ ํ•œ๋‹ค.
  • HF ์‹ค์ œ ์—ฐ๋™์— ํ•„์š”ํ•œ ํ™˜๊ฒฝ๋ณ€์ˆ˜ ๋ณ€๊ฒฝ ์‚ฌํ•ญ์€ ํŒ€ Discord์— ๊ณต์œ ํ•ด์•ผ ํ•œ๋‹ค.
    • HF_EXTRACTION_ENDPOINT_URL
    • HF_EXTRACTION_API_TOKEN
    • HF_EXTRACTION_MODEL_NAME

โœ… ์™„๋ฃŒ ์กฐ๊ฑด

  • job_results.extraction_result ์ปฌ๋Ÿผ ์ถ”๊ฐ€ migration์ด ํฌํ•จ๋˜์–ด ์žˆ๋‹ค.
  • worker๊ฐ€ ์ƒ์„ฑํ•œ extraction_result๊ฐ€ JobRepository.upsert_job_result()๋ฅผ ํ†ตํ•ด DB์— ์ €์žฅ๋œ๋‹ค.
  • JobRepository.get_job_result()๊ฐ€ ์ €์žฅ๋œ extraction_result๋ฅผ ์ •์ƒ ์กฐํšŒํ•œ๋‹ค.
  • /jobs/{jobId}/result ์‘๋‹ต์— extraction_result๊ฐ€ ํฌํ•จ๋œ๋‹ค.
  • extraction_result๊ฐ€ null์ธ ๊ฒฝ์šฐ์—๋„ ๊ธฐ์กด job result ํ๋ฆ„์ด ๊นจ์ง€์ง€ ์•Š๋Š”๋‹ค.
  • ๊ด€๋ จ ํ…Œ์ŠคํŠธ๊ฐ€ ์ถ”๊ฐ€๋˜๊ณ  ์ „์ฒด ํ…Œ์ŠคํŠธ๊ฐ€ ํ†ต๊ณผํ•œ๋‹ค.

Metadata

Metadata

Assignees

No one assigned

    Labels

    feat์ƒˆ๋กœ์šด ๊ธฐ๋Šฅ ์ถ”๊ฐ€

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions