Daily Tech Briefing
AI 科技速览
每天 5 分钟内学习 AI。获取最新的人工智能新闻,理解其重要性,并学习如何将其应用于您的工作。
Dev.to AI · 2026/8/5 03:21:58
Image Upload Moderation Beyond Node.js: Classify NSFW and Violence with Multimodal Chat
AI 中文解读
GPT-4o的进阶玩法来了!这次不是单纯识别图片,而是让AI用“聊天”的方式给图片内容下结论,比如判断一张图是否涉及色情或暴力。最妙的是,它严格按预设的“评分表”输出结果,比如“暴力程度:高”,而不是含糊地说“这图有点危险”。这样一来,审核结果能直接入库、复查,出错了还能追溯原因。
对普通人来说,这意味着你上传照片到社交平台或论坛时,AI审核会更靠谱、更透明。以前“一刀切”的误删情况会减少,比如历史纪录片里的战争画面就不会被当成违规内容。同时,审核成本更可控——开发者会优化流程,避免浪费算力,最终让平台的审核更高效,用户上传正常图片也不容易被误伤。简单说,AI不再只是“看”,而是学会了“讲道理”,让网络环境更安全,也让你的发言更自由。
<p>Use multimodal chat with a strict JSON schema when your policy needs explainable labels for uploaded images; otherwise reach for a managed, fixed-taxonomy service. There is no dedicated image moderation endpoint here, so the practical design is a policy prompt, a vision-capable chat model, schema validation, and a conservative fallback.</p>
<p>That is my short answer. I would not ship the model's prose directly into an allow/block decision. I keep the original decision for audits, translate it into a small internal status, and make the eval set the release gate. The model is one component of the policy system — not the policy system itself.</p>
<h2>
What should a Python image upload moderation example classify for NSFW and violence?
</h2>
<p>The categories should come from the app's actual rules. For a general user-content product, I start with nudity, graphic violence, hate symbols, drugs, and minors-risk. I don't pretend those labels are universal: a medical forum and a marketplace need different thresholds, and a historical archive may legitimately show symbols that a profile-photo product should reject.</p>
<p>My first notebook pass is deliberately boring. I assemble a small set of allowed, blocked, and ambiguous pictures; write the expected category labels; and record the policy reason in plain English. Then I run the same prompt and schema across every candidate model. The score I care about first is false negatives on the block set, followed by false positives on harmless uploads. Overall accuracy can hide both.</p>
<p>This is also where a JSON schema earns its keep. A response containing <code>"graphic_violence": "high"</code> can be validated, stored, and compared. A paragraph such as “this appears concerning” can't reliably drive a queue or an appeal. Keep the provider response beside a normalized status such as <code>allow</code>, <code>review</code>, or <code>block</code>; when policy changes, you can replay the raw decisions without migrating every old record.</p>
<p>I learned the cost side the annoying way: one evaluation run consumed 18.7 million input tokens, roughly 3.4 times my estimate, because I had repeated the full policy rubric for every crop and retry. My notebook showed a reasonable per-case estimate, but the production-shaped harness expanded each source into several variants, then retried cases whose structured response failed validation. I had measured the neat path and budgeted for the messy one. I stopped the run, grouped usage by fixture and attempt, and found that the largest images weren't the main culprit; duplicated policy text across the expanded cases was. The fix was measurement, not guesswork. I made prompt tokens a first-class eval column, deduplicated image variants before dispatch, and reported cost per accepted decision rather than cost per request. That last denominator matters because a cheap response that lands in manual review hasn't completed the job. I also put a batch-level ceiling around experiments, so a mistaken multiplier stops early instead of becoming a surprise at the end of the day. Now I count prompt tokens before any large run and inspect the distribution, not just its mean.</p>
<p>Small batches first.</p>
<p>No prose parsing.</p>
<h2>
The focused implementation
</h2>
<p>The example below sends one local image to the verified <code>POST /v1/chat/completions</code> route. It uses a data URL so the program is self-contained, takes both the API key and vision model ID from environment variables, requires structured JSON, and retries rate limits while honoring <code>Retry-After</code>. I use the standard-library HTTP client because this article is about the policy boundary, not a framework choice.<br>
</p>
<div class="highlight js-code-highlight">
<pre class="highlight python"><code><span class="kn">import</span> <span class="n">base64</span>
<span class="kn">import</span> <span class="n">json</span>
<span class="kn">import</span> <span class
分享
阅读原文 ↗