Browser Run - Crawl endpoint now respects the Content Signals `use` directive

<p>The <a href="https://developers.cloudflare.com/browser-run/quick-actions/crawl-endpoint/"><code>/crawl</code></a> endpoint now respects the <code>use</code> directive of the <a href="https://contentsignals.org/" target="_blank" rel="noopener">Content Signals<span class="external-link"> ↗︎</span></a> standard, letting site owners express the maximum level at which their content may be used.</p> <p>You can declare your intended level with the new <code>contentUse</code> parameter. Allowed values, from least to most permissive, are <code>reference</code> and <code>full</code>, and the default is <code>full</code>. If a target site's <code>robots.txt</code> sets a <code>use</code> level that is more restrictive than your declared <code>contentUse</code>, the crawl request is rejected with a <code>400</code> error.</p> <figure class="nb-code-figure" data-nb-lang="bash"><pre class="astro-code astro-code-themes github-light github-dark nb-shiki-c6xiwz" tabindex="0" data-language="bash" data-nb-lang="bash"><code><span class="line"><span class="nb-shiki-1t8gfj">curl</span><span class="nb-shiki-dzsirb"> -X</span><span class="nb-shiki-mdbnqw"> POST</span><span class="nb-shiki-mdbnqw"> 'https://api.cloudflare.com/client/v4/accounts/{account_id}/browser-rendering/crawl'</span><span class="nb-shiki-dzsirb"> \</span></span> <span class="line"><span class="nb-shiki-dzsirb"> -H</span><span class="nb-shiki-mdbnqw"> 'Authorization: Bearer &lt;apiToken&gt;'</span><span class="nb-shiki-dzsirb"> \</span></span> <span class="line"><span class="nb-shiki-dzsirb"> -H</span><span class="nb-shiki-mdbnqw"> 'Content-Type: application/json'</span><span class="nb-shiki-dzsirb"> \</span></span> <span class="line"><span class="nb-shiki-dzsirb"> -d</span><span class="nb-shiki-mdbnqw"> '{</span></span> <span class="line"><span class="nb-shiki-mdbnqw"> "url": "https://example.com",</span></span> <span class="line"><span class="nb-shiki-mdbnqw"> "contentUse": "reference",</span></span> <span class="line"><span class="nb-shiki-mdbnqw"> "formats": ["markdown"]</span></span> <span class="line"><span class="nb-shiki-mdbnqw"> }'</span></span></code></pre></figure> <p>For more information, refer to <a href="https://developers.cloudflare.com/browser-run/quick-actions/crawl-endpoint/#content-signals">Content Signals</a> in the <code>/crawl</code> endpoint documentation.</p>