feat(ai): add image_search_query param to add_image_placeholder_v0/v1

Without an explicit query, the auto-search pipeline can only fall back
to the placeholder's `label` (often unset for context-rich cards) or
finally a generic "placeholder" string — both produce off-topic stock
photos instead of, e.g., burger / sushi shots for a food-app brief.

Builders (`buildImagePlaceholder`, `buildImagePlaceholderV1`) now accept
an optional `image_search_query` param (snake_case to match the rest of
the params interface). When set, it gets stamped onto the resulting
frame as `imageSearchQuery` — the same camelCase field
`image-search-pipeline.ts::extractQueryForNode` already prefers over
`name` and the label child.

Tool definitions in `element-tool-defs-ext-2.ts` (v0) and
`element-tool-defs-ext-6.ts` (v1) expose the new property with a
description that nudges callers to pass 2-3 keywords ("burger fries",
"modern office workspace") for product / restaurant / hero contexts.

3 new tests in `add-image-placeholder-v0.test.ts`: query stamps onto
frame, omitted query leaves field undefined, empty-string query is
treated as missing.
This commit is contained in:
Fini 2026-05-05 04:01:51 +08:00
parent f7e776de4f
commit 71d3d8b5b7
5 changed files with 74 additions and 4 deletions

View file

@ -25,6 +25,14 @@ export interface ImagePlaceholderV1Params {
* - `'system'`: emits `$color-*` refs for all fill fields.
*/
theme?: V1Theme;
/**
* 2-3 English keywords to search for when the auto-search pass replaces
* this placeholder with a real photo (e.g. "burger fries", "modern
* office"). Stamped onto the frame as `imageSearchQuery`; consumed by
* the web app's `image-search-pipeline`. Without it the pipeline falls
* back to `label` then a generic "placeholder" query.
*/
image_search_query?: string;
}
/**
@ -73,7 +81,7 @@ export function buildImagePlaceholderV1(params: ImagePlaceholderV1Params): Eleme
});
}
return {
const frame: ElementTree = {
type: 'frame',
name: 'Image Placeholder',
role: 'image-placeholder',
@ -87,4 +95,9 @@ export function buildImagePlaceholderV1(params: ImagePlaceholderV1Params): Eleme
fill: [{ type: 'solid', color: frameBg }],
children,
};
if (typeof params.image_search_query === 'string' && params.image_search_query.length > 0) {
(frame as ElementTree & { imageSearchQuery?: string }).imageSearchQuery =
params.image_search_query;
}
return frame;
}

View file

@ -20,6 +20,14 @@ export interface ImagePlaceholderParams {
* placeholder, but `add_avatar_v0` is a cleaner fit for that.
*/
corner_radius?: number;
/**
* 2-3 English keywords to search for when the auto-search pass replaces
* this placeholder with a real photo (e.g. "burger fries", "modern
* office"). Stamped onto the frame as `imageSearchQuery`; consumed by
* the web app's `image-search-pipeline`. Without it the pipeline falls
* back to `label` then a generic "placeholder" query.
*/
image_search_query?: string;
}
/**
@ -72,7 +80,7 @@ export function buildImagePlaceholder(params: ImagePlaceholderParams): ElementTr
});
}
return {
const frame: ElementTree = {
type: 'frame',
name: 'Image Placeholder',
role: 'image-placeholder',
@ -86,4 +94,9 @@ export function buildImagePlaceholder(params: ImagePlaceholderParams): ElementTr
fill: [{ type: 'solid', color: '#F1F5F9' }],
children,
};
if (typeof params.image_search_query === 'string' && params.image_search_query.length > 0) {
(frame as ElementTree & { imageSearchQuery?: string }).imageSearchQuery =
params.image_search_query;
}
return frame;
}

View file

@ -108,4 +108,28 @@ describe('add_image_placeholder_v0', () => {
);
expect(await readFile(fp, 'utf-8')).toBe(before);
});
it('image_search_query is stamped onto the frame as imageSearchQuery', async () => {
const fp = await fresh('a.op');
await handleAddImagePlaceholderV0({
filePath: fp,
image_search_query: 'burger fries',
});
const ph = getRoot(await readDoc(fp));
expect(ph.imageSearchQuery).toBe('burger fries');
});
it('omitting image_search_query leaves the field undefined', async () => {
const fp = await fresh('a.op');
await handleAddImagePlaceholderV0({ filePath: fp });
const ph = getRoot(await readDoc(fp));
expect(ph.imageSearchQuery).toBeUndefined();
});
it('empty-string image_search_query is treated as missing', async () => {
const fp = await fresh('a.op');
await handleAddImagePlaceholderV0({ filePath: fp, image_search_query: '' });
const ph = getRoot(await readDoc(fp));
expect(ph.imageSearchQuery).toBeUndefined();
});
});

View file

@ -206,7 +206,9 @@ export const ELEMENT_TOOL_DEFINITIONS_EXT_2 = [
'The "this will be an image later" affordance. Use for "photo slot", "hero image area", ' +
'"upload zone", "cover placeholder", "图片占位". Separate from G() (which fetches real ' +
'images via search) — this emits a visual placeholder frame, not an image node. ' +
'schemaVersion 1.0',
'When the canvas runs its auto-search pass, an image_search_query (or label) lets it ' +
'fetch a relevant photo to replace the gray box; without one it falls back to a ' +
'generic stock photo. schemaVersion 1.0',
inputSchema: {
type: 'object' as const,
properties: {
@ -226,6 +228,14 @@ export const ELEMENT_TOOL_DEFINITIONS_EXT_2 = [
type: 'number',
description: 'Corner radius (default 8). Use larger for card-style, 0 for sharp.',
},
image_search_query: {
type: 'string',
description:
'2-3 English keywords for the auto-search pass to fetch a photo with ' +
'(e.g. "burger fries", "modern office workspace", "yoga sunset"). Strongly ' +
'recommended for restaurant cards, product photos, hero banners, etc — ' +
'without it the pipeline searches for the label or a generic placeholder.',
},
parent_id: parentIdProp,
pageId: pageIdProp,
},

View file

@ -388,7 +388,9 @@ export const ELEMENT_TOOL_DEFINITIONS_EXT_6 = [
'Theme-aware image placeholder (v1). theme="light" (default): byte-parity with ' +
'add_image_placeholder_v0. theme="dark": bg → bgDeep, icon → textMuted, label → textMuted. ' +
'theme="system": $color-* refs. Gray box + centered icon + optional caption. ' +
'schemaVersion 1.0',
'When the canvas runs its auto-search pass, an image_search_query (or label) lets ' +
'it fetch a relevant photo to replace the gray box; without one it falls back to a ' +
'generic stock photo. schemaVersion 1.0',
inputSchema: {
type: 'object' as const,
properties: {
@ -404,6 +406,14 @@ export const ELEMENT_TOOL_DEFINITIONS_EXT_6 = [
enum: ['light', 'dark', 'system'],
description: 'Theme variant. Default "light".',
},
image_search_query: {
type: 'string',
description:
'2-3 English keywords for the auto-search pass to fetch a photo with ' +
'(e.g. "burger fries", "modern office workspace", "yoga sunset"). Strongly ' +
'recommended for restaurant cards, product photos, hero banners, etc — ' +
'without it the pipeline searches for the label or a generic placeholder.',
},
parent_id: parentIdProp,
pageId: pageIdProp,
},