llm-html

CMDOP Skill — 설치 및 사용을위한 CMDOP 에이전트:

Bash

cmdop-skill install llm-html

Loading code...

LLM-optimized HTML 청소: 수분 추출, 토큰 예산, 다중 출력 형식.

설치

Bash

pip install llm-html

Loading code...

빠른 시작

Python

from llm_html import HTMLCleaner, CleanerConfig, OutputFormat

# 기본 청소
  cleaner = HTMLCleaner()
  result = cleaner.clean(html)
  print(f"Reduction: {result.stats.reduction_percent}%")

# 수분-첫 번째 (Next.js, Nuxt 등에서 SSR 데이터 추출)
  if result.hydration_data:
    data = result.hydration_data
  else:
    cleaned = result.html

Loading code...

편의 함수

Python

from llm_html import clean, clean_to_json, clean_html, clean_for_llm

# 빠른 청소
  result = clean(html)

# JSON을 가져오기 (SSR 데이터가 있으면, 그렇지 않으면 청소된 HTML)
  data = clean_to_json(html)

# 파이프라인에 대한 전체 제어
  result = clean_html(html, max_tokens=5000)
  result = clean_for_llm(html, output_format="markdown")

Loading code...

출력 형식

Python

from llm_html import to_markdown, to_aom_yaml, to_xtree

md = to_markdown(html)
  aom = to_aom_yaml(html)
  xtree = to_xtree(html)

Loading code...

다운 샘플링

토큰 예산 목표화에 D2Snap 알고리즘:

Python

from llm_html import downsample_html, estimate_tokens

  tokens = estimate_tokens(html)
  if tokens > 10000:
    html = downsample_html(html, target_tokens=8000)

Loading code...

의미적 분할

대형 페이지를 LLM 크기의 조각으로 분할:

Python

from llm_html import SemanticChunker, ChunkConfig

  config = ChunkConfig(max_tokens=8000, max_items=20)
  chunker = SemanticChunker(config)
  result = chunker.chunk(soup)
  for chunk in result.chunks:
    process(chunk.html)

Loading code...

Shadow DOM

Web Component를 LLM에 대한 비전을 위해 평평하게:

Python

from llm_html import flatten_shadow_dom

  flat = flatten_shadow_dom(html)

Loading code...

도우미

Python

from llm_html import html_to_text, extract_links, extract_images, json_to_toon

  text = html_to_text(html)
  links = extract_links(html, base_url="https://example.com")
  images = extract_images(html)
  toon = json_to_toon({"key": "value"})

Loading code...

라이선스

MIT

Bash

cmdop-skill install llm-html

Bash

pip install llm-html

Python

from llm_html import HTMLCleaner, CleanerConfig, OutputFormat

# 기본 청소
  cleaner = HTMLCleaner()
  result = cleaner.clean(html)
  print(f"Reduction: {result.stats.reduction_percent}%")

# 수분-첫 번째 (Next.js, Nuxt 등에서 SSR 데이터 추출)
  if result.hydration_data:
    data = result.hydration_data
  else:
    cleaned = result.html

Python

from llm_html import clean, clean_to_json, clean_html, clean_for_llm

# 빠른 청소
  result = clean(html)

# JSON을 가져오기 (SSR 데이터가 있으면, 그렇지 않으면 청소된 HTML)
  data = clean_to_json(html)

# 파이프라인에 대한 전체 제어
  result = clean_html(html, max_tokens=5000)
  result = clean_for_llm(html, output_format="markdown")

Python

from llm_html import to_markdown, to_aom_yaml, to_xtree

md = to_markdown(html)
  aom = to_aom_yaml(html)
  xtree = to_xtree(html)

Python

from llm_html import downsample_html, estimate_tokens

  tokens = estimate_tokens(html)
  if tokens > 10000:
    html = downsample_html(html, target_tokens=8000)

Python

from llm_html import SemanticChunker, ChunkConfig

  config = ChunkConfig(max_tokens=8000, max_items=20)
  chunker = SemanticChunker(config)
  result = chunker.chunk(soup)
  for chunk in result.chunks:
    process(chunk.html)

Python

from llm_html import flatten_shadow_dom

  flat = flatten_shadow_dom(html)

Python

from llm_html import html_to_text, extract_links, extract_images, json_to_toon

  text = html_to_text(html)
  links = extract_links(html, base_url="https://example.com")
  images = extract_images(html)
  toon = json_to_toon({"key": "value"})

llm-html

llm-html

설치

빠른 시작

편의 함수

출력 형식

다운 샘플링

의미적 분할

Shadow DOM

도우미

라이선스

리뷰