Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
cb3dd19
feat: 유사도 torch 삭제 후 onnx 추가
thkim7 Sep 11, 2025
72002de
feat: 유사도 torch 삭제 후 onnx 추가
thkim7 Sep 11, 2025
99e2f99
chore: onnx 모델 저장용 볼륨 docker-composer에 추가
thkim7 Sep 11, 2025
3f2a398
Merge branch 'develop' into feature/onnx
thkim7 Sep 11, 2025
cab225b
chore: poetry.lock 추가
thkim7 Sep 12, 2025
c5ac0ab
chore: poetry.lock 추가
thkim7 Sep 12, 2025
000f524
chore: poetry run black & ci-python 원상 복구
thkim7 Sep 12, 2025
78c43d8
chore: ci-python 머지 테스트 추가
thkim7 Sep 12, 2025
219a5b6
Merge pull request #87 from Kernel180-BE12/feature/onnx
thkim7 Sep 12, 2025
be53bb4
typeHandler가 등록되지 않는 버그 (#88)
can019 Sep 13, 2025
5579b7c
기본적인 global exception handling (#90)
can019 Sep 13, 2025
34c22dc
Caddyfile copy 추가 (#89)
can019 Sep 13, 2025
1f38926
refactor: 필요없는 API url 삭제
dktkf Sep 14, 2025
d60939f
refactor: 2차 마일스톤을위한 공통 request및 response 주석
dktkf Sep 14, 2025
e8c83c3
refactor:
dktkf Sep 14, 2025
b8e3aaa
Merge pull request #92 from Kernel180-BE12/feature/milestone_patch
kakusiA Sep 14, 2025
959adf4
Workflow dummy data insert sql (alpha) (#93)
can019 Sep 15, 2025
7a147bd
chore: v0.0.4 erd 적용
can019 Sep 15, 2025
1050dd3
Loki, Grafana 로컬 세팅 및 log 처리 (#95)
can019 Sep 15, 2025
91db5c3
로그아웃 api (#96)
bwnfo3 Sep 15, 2025
e36e407
Grafana Loki 로깅 (develop) (#97)
can019 Sep 15, 2025
71f3b5b
Pagination dto 및 공통 service (#94)
can019 Sep 15, 2025
d766df4
feat: crawling_util 한개로 합침
thkim7 Sep 15, 2025
5a9dd2b
Merge branch 'develop' of https://github.com/Kernel180-BE12/Final-4te…
thkim7 Sep 16, 2025
1106b30
Merge branch 'develop' of https://github.com/Kernel180-BE12/Final-4te…
thkim7 Sep 16, 2025
a3f4bb7
블로그 자동화 배치 Job 및 Tasklet 세팅 (#91)
jihukimme Sep 16, 2025
1acd33c
chore: 코드에 있는 job_id 전부 제거
thkim7 Sep 16, 2025
5700b77
chore: poetry run black .
thkim7 Sep 16, 2025
2172cde
Loki e2e test support (#99)
can019 Sep 16, 2025
edcbbce
Merge pull request #98 from Kernel180-BE12/feature/crawl_service_refa…
thkim7 Sep 16, 2025
c4759de
Merge branch 'develop' of https://github.com/Kernel180-BE12/Final-4te…
thkim7 Sep 16, 2025
af62f52
feat: 예시 데이터로 GPT한테 콘텐츠 생성 후 blogger에 업로드 성공
thkim7 Sep 16, 2025
64c78c7
feat: rag로 콘텐츠 생성 후 blogger 포스팅 테스트 통과
thkim7 Sep 16, 2025
4bffbae
feat: poetry run black
thkim7 Sep 16, 2025
c57c5ae
Merge pull request #102 from Kernel180-BE12/feature/rag_draft
thkim7 Sep 16, 2025
a1d7768
RDB Log4j2 append (experimental) (#104)
can019 Sep 16, 2025
da3647d
refactor : FAST API Response 리펙토링 (#105)
kakusiA Sep 17, 2025
2cc0c5d
test: spring Fastapi testing
dktkf Sep 17, 2025
a0d1bbb
Merge remote-tracking branch 'origin/develop' into feature/spring-fas…
dktkf Sep 17, 2025
7bfab7f
refactor: test-url response refactor
dktkf Sep 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: CI (Python/FastAPI)

on:
push:
branches:
- feature/onnx
tags:
- 'pre-processing-v*'
pull_request:
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/deploy-java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,16 @@ jobs:
target: "~/app/docker/production/"
overwrite: true

- name: Copy Caddyfile to EC2
uses: appleboy/scp-action@v0.1.7
with:
host: ${{ secrets.SERVER_HOST }}
username: ubuntu
key: ${{ secrets.SERVER_SSH_KEY }}
source: "docker/production/Caddyfile"
target: "~/app/docker/production/"
overwrite: true

- name: Deploy on EC2
uses: appleboy/ssh-action@v1.0.3
with:
Expand Down
37 changes: 15 additions & 22 deletions apps/pre-processing-service/app/api/endpoints/blog.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,14 @@
from ...model.schemas import *
from app.service.blog.tistory_blog_post_service import TistoryBlogPostService
from app.service.blog.naver_blog_post_service import NaverBlogPostService
from ...service.blog.blogger_blog_post_service import BloggerBlogPostService
from ...service.blog.blogger_blog_post_adapter import (
BloggerBlogPostAdapter,
) # 수정된 import
from app.utils.response import Response

router = APIRouter()


@router.get("/", summary="블로그 API 상태 확인")
async def root():
return {"message": "blog API"}


@router.post(
"/rag/create",
response_model=ResponseBlogCreate,
Expand All @@ -39,50 +37,45 @@ async def publish(request: RequestBlogPublish):
"""
if request.tag == "naver":
naver_service = NaverBlogPostService()
result = naver_service.post_content(
response_data = naver_service.post_content(
title=request.post_title,
content=request.post_content,
tags=request.post_tags,
)

if not result:
if not response_data:
raise CustomException(
"네이버 블로그 포스팅에 실패했습니다.", status_code=500
)
return ResponseBlogPublish(
job_id=1, schedule_id=1, schedule_his_id=1, status="200", metadata=result
)

return Response.ok(response_data)

elif request.tag == "tistory":
tistory_service = TistoryBlogPostService()
result = tistory_service.post_content(
response_data = tistory_service.post_content(
title=request.post_title,
content=request.post_content,
tags=request.post_tags,
)

if not result:
if not response_data:
raise CustomException(
"티스토리 블로그 포스팅에 실패했습니다.", status_code=500
)

return ResponseBlogPublish(
job_id=1, schedule_id=1, schedule_his_id=1, status="200", metadata=result
)
return Response.ok(response_data)

elif request.tag == "blogger":
blogger_service = BloggerBlogPostService()
result = blogger_service.post_content(
blogger_service = BloggerBlogPostAdapter() # 수정: Adapter 사용
response_data = blogger_service.post_content(
title=request.post_title,
content=request.post_content,
tags=request.post_tags,
)

if not result:
if not response_data:
raise CustomException(
"블로거 블로그 포스팅에 실패했습니다.", status_code=500
)

return ResponseBlogPublish(
job_id=1, schedule_id=1, schedule_his_id=1, status="200", metadata=result
)
return Response.ok(response_data)
23 changes: 0 additions & 23 deletions apps/pre-processing-service/app/api/endpoints/keywords.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,6 @@
router = APIRouter()


@router.get("/", summary="키워드 API 상태 확인")
async def root():
"""
키워드 API가 정상 동작하는지 확인
"""
return {"message": "keyword API"}


@router.post(
"/search", response_model=ResponseNaverSearch, summary="네이버 키워드 검색"
)
Expand All @@ -23,9 +15,6 @@ async def search(request: RequestNaverSearch):

요청 예시:
{
"job_id": 1,
"schedule_id": 1,
"schedule_his_id": 1,
"tag": "naver",
"category": "50000000",
"start_date": "2025-09-01",
Expand All @@ -34,15 +23,3 @@ async def search(request: RequestNaverSearch):
"""
response_data = await keyword_search(request)
return response_data


@router.post(
"/ssadagu/validate",
response_model=ResponseNaverSearch,
summary="사다구몰 키워드 검증",
)
async def ssadagu_validate(request: RequestNaverSearch):
"""
사다구몰 키워드 검증 테스트용 엔드포인트
"""
return ResponseNaverSearch()
71 changes: 32 additions & 39 deletions apps/pre-processing-service/app/api/endpoints/product.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from ...service.crawl_service import CrawlService
from ...service.search_service import SearchService
from ...service.match_service import MatchService
from ...service.similarity_service import SimilarityService


# from ...service.similarity_service import SimilarityService
Expand All @@ -16,27 +17,19 @@
router = APIRouter()


@router.get("/", summary="상품 API 상태 확인")
async def root():
"""
상품 API 서버 상태 확인용 엔드포인트
"""
return {"message": "product API"}


@router.post("/search", response_model=ResponseSadaguSearch, summary="상품 검색")
async def search(request: RequestSadaguSearch):
"""
요청된 키워드로 사다구몰 상품을 검색합니다.
"""
try:
search_service = SearchService()
result = await search_service.search_products(request)
response_data = await search_service.search_products(request)

if not result:
if not response_data:
raise CustomException(500, "상품 검색에 실패했습니다.", "SEARCH_FAILED")

return result
return response_data
except InvalidItemDataException as e:
raise HTTPException(status_code=e.status_code, detail=e.detail)
except Exception as e:
Expand All @@ -50,56 +43,56 @@ async def match(request: RequestSadaguMatch):
"""
try:
match_service = MatchService()
result = match_service.match_products(request)
response_data = match_service.match_products(request)

if not result:
if not response_data:
raise CustomException(500, "상품 매칭에 실패했습니다.", "MATCH_FAILED")

return result
return response_data
except InvalidItemDataException as e:
raise HTTPException(status_code=e.status_code, detail=e.detail)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))


# @router.post(
# "/similarity", response_model=ResponseSadaguSimilarity, summary="상품 유사도 분석"
# )
# async def similarity(request: RequestSadaguSimilarity):
# """
# 매칭된 상품들 중 키워드와의 유사도를 계산하여 최적의 상품을 선택합니다.
# """
# try:
# similarity_service = SimilarityService()
# result = similarity_service.select_product_by_similarity(request)
#
# if not result:
# raise CustomException(
# 500, "유사도 분석에 실패했습니다.", "SIMILARITY_FAILED"
# )
#
# return result
# except InvalidItemDataException as e:
# raise HTTPException(status_code=e.status_code, detail=e.detail)
# except Exception as e:
# raise HTTPException(status_code=500, detail=str(e))
@router.post(
"/similarity", response_model=ResponseSadaguSimilarity, summary="상품 유사도 분석"
)
async def similarity(request: RequestSadaguSimilarity):
"""
매칭된 상품들 중 키워드와의 유사도를 계산하여 최적의 상품을 선택합니다.
"""
try:
similarity_service = SimilarityService()
response_data = similarity_service.select_product_by_similarity(request)

if not response_data:
raise CustomException(
500, "유사도 분석에 실패했습니다.", "SIMILARITY_FAILED"
)

return response_data
except InvalidItemDataException as e:
raise HTTPException(status_code=e.status_code, detail=e.detail)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))


@router.post(
"/crawl", response_model=ResponseSadaguCrawl, summary="상품 상세 정보 크롤링"
)
async def crawl(request: Request, body: RequestSadaguCrawl):
async def crawl(body: RequestSadaguCrawl):
"""
상품 상세 페이지를 크롤링하여 상세 정보를 수집합니다.
"""
try:
crawl_service = CrawlService()
result = await crawl_service.crawl_product_detail(body)
response_data = await crawl_service.crawl_product_detail(body)

if not result:
if not response_data:
raise CustomException(500, "상품 크롤링에 실패했습니다.", "CRAWL_FAILED")

return result
return response_data
except InvalidItemDataException as e:
raise HTTPException(status_code=e.status_code, detail=e.detail)
except ItemNotFoundException as e:
Expand Down
45 changes: 45 additions & 0 deletions apps/pre-processing-service/app/api/endpoints/sample.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
from fastapi import APIRouter
from ...model.schemas import *
from app.utils.response import Response
router = APIRouter()


@router.get("/")
async def root():
return {"message": "sample API"}


@router.post("/keywords/search", summary="네이버 키워드 검색")
async def search(request: RequestNaverSearch):
return Response.ok({"test" : "hello world"})



@router.post("/blogs/rag/create", summary="RAG 기반 블로그 콘텐츠 생성")
async def rag_create(request: RequestBlogCreate):
return Response.ok({"test": "hello world"})


@router.post("/blogs/publish", summary="블로그 콘텐츠 배포")
async def publish(request: RequestBlogPublish):
return Response.ok({"test" : "hello world"})


@router.post("/products/search", summary="상품 검색")
async def product_search(request: RequestSadaguSearch):
return Response.ok({"test" : "hello world"})


@router.post("/products/match", summary="상품 매칭")
async def product_match(request: RequestSadaguMatch):
return Response.ok({"test" : "hello world"})


@router.post("/products/similarity", summary="상품 유사도 분석")
async def product_similarity(request: RequestSadaguSimilarity):
return Response.ok({"test" : "hello world"})


@router.post("/products/crawl", summary="상품 상세 정보 크롤링")
async def product_crawl(request: RequestSadaguCrawl):
return Response.ok({"test": "hello world"})
18 changes: 4 additions & 14 deletions apps/pre-processing-service/app/api/endpoints/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,6 @@
router = APIRouter()


@router.get("/")
async def root():
return {"message": "테스트 API"}


@router.get("/hello/{name}", tags=["hello"])
# @log_api_call
async def say_hello(name: str):
Expand Down Expand Up @@ -67,19 +62,14 @@ def with_meta(data: Mapping[str, Any], meta: Mapping[str, Any]) -> Dict[str, Any

@router.get("/tester", response_model=None)
async def processing_tester():
meta = {
"job_id": 1,
"schedule_id": 1,
"schedule_his_id": 1, # ✅ 타이포 수정
}
request_dict = {
"tag": "naver",
"category": "50000000",
"start_date": "2025-09-01",
"end_date": "2025-09-02",
}
# 네이버 키워드 검색
naver_request = RequestNaverSearch(**with_meta(meta, request_dict))
naver_request = RequestNaverSearch(**with_meta(request_dict))
response_data = await keyword_search(naver_request)
keyword = response_data.get("keyword")
loguru.logger.info(keyword)
Expand All @@ -89,21 +79,21 @@ async def processing_tester():
}

# 싸다구 상품 검색
sadagu_request = RequestSadaguSearch(**with_meta(meta, keyword))
sadagu_request = RequestSadaguSearch(**with_meta(keyword))
search_service = SearchService()
keyword_result = await search_service.search_products(sadagu_request)
loguru.logger.info(keyword_result)

# 싸다구 상품 매치
keyword["search_results"] = keyword_result.get("search_results")
keyword_match_request = RequestSadaguMatch(**with_meta(meta, keyword))
keyword_match_request = RequestSadaguMatch(**with_meta(keyword))
match_service = MatchService()
keyword_match_response = match_service.match_products(keyword_match_request)
loguru.logger.info(keyword_match_response)

# 싸다구 상품 유사도 분석
keyword["matched_products"] = keyword_match_response.get("matched_products")
keyword_similarity_request = RequestSadaguSimilarity(**with_meta(meta, keyword))
keyword_similarity_request = RequestSadaguSimilarity(**with_meta(keyword))
# similarity_service = SimilarityService()
# keyword_similarity_response = similarity_service.select_product_by_similarity(
# keyword_similarity_request
Expand Down
3 changes: 2 additions & 1 deletion apps/pre-processing-service/app/api/router.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# app/api/router.py
from fastapi import APIRouter
from .endpoints import keywords, blog, product, test
from .endpoints import keywords, blog, product, test,sample
from ..core.config import settings

api_router = APIRouter()
Expand All @@ -17,6 +17,7 @@
# 모듈 테스터를 위한 endpoint -> 추후 삭제 예정
api_router.include_router(test.router, prefix="/tests", tags=["Test"])

api_router.include_router(sample.router, prefix="/v0", tags=["Sample"])

@api_router.get("/ping")
async def root():
Expand Down
Loading
Loading