-
-
Notifications
You must be signed in to change notification settings - Fork 65
JSON API
Enfugue uses a JSON API in it's frontend that can also be used by developers to integrate the Enfugue engine and other management capabilities using a simple HTTP interface.
Want to see how to make images? Go to Invoke.
When using Enfugue in no-authentication mode, there is no authentication necessary, and all operations can be performed without logging in.
When using Authentication, all endpoints require you pass an authentication token. To obtain one, send a POST
request to /api/login
with the credentials of a user. For example:
$ curl -X POST -d '{"username": "enfugue", "password": "enfugue"}' https://app.enfugue.ai:45554/api/login
{
"meta": {
"params": {}
},
"data": {
"type": "AuthenticationToken",
"attributes": {
"access_token": "3a12f47da5da41c798ef0c60bbf00ce0",
"created": "2023-08-12T00:04:54.515414",
"expires": "2023-09-11T10:04:54.515414",
"id": 10,
"token_type": "Bearer",
"user_id": 1
}
}
}
To pass the authentication token in your request. You should use the token_type
along with the access_token
as an Authentication
header. For example:
$ curl https://app.enfugue.ai:45554/api
{
"errors": [
{
"status": "401",
"title": "AuthenticationError",
"detail": "Invalid or no credentials supplied."
}
]
}
$ curl -H "Authorization: Bearer 3a12f47da5da41c798ef0c60bbf00ce0" https://app.enfugue.ai:45554/api
{
"meta": {
"params": {}
},
"data": {
"status": "idle",
"system": {
"downloads": {
"active": 0,
"queued": 0,
"total": 0
},
"invocations": {
"active": false,
"queued": 0,
"total": 0
}
},
"gpu": {
"driver": "536.99",
"name": "NVIDIA GeForce RTX 3090 Ti",
"load": 0.43,
"temp": 56,
"memory": {
"free": 23014,
"total": 24564,
"used": 1550,
"util": 0.06310047223579221
}
},
"version": "0.0.0",
"uptime": 199.639343
}
}
There are two levels of users, admin
and user
. Admins are able to perform any action, whereas users are only able to invoke the engine and view their own invocation history.
This documentation serves as a starting point for your own integrations. If you want to see more examples of API calls, it's recommended to simply monitor your network panel while using the UI. You can send the same network requests as the UI does from anywhere else.
Persistent database objects may be manipulated in a standard RESTful way. You can retrieve objects using the GET
HTTP method, create them with POST
, modify them with PATCH
, and delete them with DELETE
.
These objects include:
- Models (pre-configured sets of checkpoints, lora, etc.)
- Users (only relevant when using authentication)
- Invocation History (GET and DELETE only)
The endpoints will be enumerated later in this document.
When using the POST
or PATCH
methods, you may pass a body of parameters using JSON that match the properties of the intended objects.
When using the GET
method, there are a number of URL parameters you may pass to perform various operations to manipulate what and how the data is retrieved. These parameters include:
-
limit
andoffset
- set these to integer numbers to control how many items to retrieve, and an offset to allow you to iteratively retrieve the next results (page). For example,?limit=10
will retrieve the first 10 results, and?limit=10&offset=10
will retrieve the next 10. -
sort
- Pass one or more of these to control how the data you're looking for is sorted. For example,?sort=name
to sort by name in an ascending direction, and?sort=name:desc
to sort in a descending direction. Pass multiplesort
values by simply adding them to the parameter set, i.e.?sort=name&sort=size
-
ilike
- Pass one or more of these for an easy search on a parameter. For example, pass?ilike=name:real
will find all objects where thename
parameter contains the textreal
(case insensitive). -
filter
- Pass one or more of these for a more specific search on a parameter. For example, pass?filter=name:Realistic
for a case-sensitive match. Pass multiple to filter by more than one column at once. Append additional values with|
to denote multiple possible values to search for, like?filter=name:Realistic|Realism
.
GET https://app.enfugue.ai:45554/api/users(/{{ username }})
Username is optional.
$ curl https://app.enfugue.ai:45554/api/users
{
"meta": {
"params": {},
"count": 1
},
"data": [
{
"type": "User",
"attributes": {
"first_name": "Default",
"id": 1,
"last_login": "2023-08-12T00:04:54.512234",
"last_name": "User",
"username": "enfugue"
}
}
]
}
POST https://app.enfugue.ai:45554/api/users
$ curl -X POST -d '{"username": "myuser", "new_password": "mypassword", "repeat_password": "mypassword", "admin": true}' https://app.enfugue.ai:45554/api/users
{
"meta": {
"params": {},
"data": {
"type": "User",
"attributes": {
"first_name": null,
"id": 2,
"last_login": null,
"last_name": null,
"username": "myuser"
}
}
}
}
PATCH https://app.enfugue.ai:45554/api/users/{{ username }}
$ curl -X POST -d '{"admin": false}' https://app.enfugue.ai:45554/api/users/myuser
{
"meta": {
"params": {},
"data": {
"type": "User",
"attributes": {
"first_name": null,
"id": 2,
"last_login": null,
"last_name": null,
"username": "myuser"
}
}
}
}
DELETE https://app.enfugue.ai:45554/api/users/{{ username }}
$ curl -X DELETE https://app.enfugue.ai:45554/api/users/myuser
{
"meta": {
"params": {}
},
"data": null
}
GET /api/models(/{{ model_name }})
$ curl https://app.enfugue.ai:45554/api/models
{
"meta": {
"params": {},
"count": 2
},
"data": [
{
"type": "DiffusionModel",
"attributes": {
"model": "realisticVisionV40_v40VAE.safetensors",
"name": "Realistic Vision",
"negative_prompt": "deformed, (((distorted face))), ((malformed face)), blurry, bad anatomy, bad eyes, disfigured, mutation, mutated, extra limb, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, out of focus, long neck, long body, mutated hands and fingers, out of frame, watermark, cut off, bad art, grainy",
"prompt": "RAW photo, highly detailed, intricate detail, 4k, 8k, sharp focus, ultra-detailed, portrait photography, cinestill 800t, Fujifilm XT3",
"size": 512
}
},
{
"type": "DiffusionModel",
"attributes": {
"model": "sd_xl_base_1.0.safetensors",
"name": "SDXL",
"negative_prompt": "deformed, (((distorted face))), ((malformed face)), blurry, bad anatomy, bad eyes, disfigured, mutation, mutated, extra limb, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, out of focus, long neck, long body, mutated hands and fingers, out of frame, watermark, cut off, bad art, grainy",
"prompt": "RAW photo, highly detailed, intricate detail, 4k, 8k, sharp focus, ultra-detailed, portrait photography, cinestill 800t, Fujifilm XT3",
"size": 1024
}
}
]
}
Include more information by passing ?include={{ object }}
, where object
is one of refiner|inpainter|scheduler|lora|lycoris|inversion|scheduler|config
$ curl https://app.enfugue.ai:45554/api/models/SDXL?include=vae\&include=refiner\&include=config
{
"meta": {
"params": {
"include": [
"vae",
"refiner",
"config"
]
},
"count": 1
},
"data": [
{
"type": "DiffusionModel",
"attributes": {
"model": "sd_xl_base_1.0.safetensors",
"name": "SDXL",
"negative_prompt": "deformed, (((distorted face))), ((malformed face)), blurry, bad anatomy, bad eyes, disfigured, mutation, mutated, extra limb, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, out of focus, long neck, long body, mutated hands and fingers, out of frame, watermark, cut off, bad art, grainy",
"prompt": "RAW photo, highly detailed, intricate detail, 4k, 8k, sharp focus, ultra-detailed, portrait photography, cinestill 800t, Fujifilm XT3",
"size": 1024
},
"include": {
"vae": [
{
"type": "DiffusionModelVAE",
"attributes": {
"diffusion_model_name": "SDXL",
"name": "xl16"
}
}
],
"refiner": [
{
"type": "DiffusionModelRefiner",
"attributes": {
"diffusion_model_name": "SDXL",
"model": "sd_xl_refiner_1.0.safetensors",
"size": 1024
}
}
],
"config": [
{
"type": "DiffusionModelDefaultConfiguration",
"attributes": {
"configuration_key": "guidance_scale",
"configuration_value": 5,
"diffusion_model_name": "SDXL"
}
},
{
"type": "DiffusionModelDefaultConfiguration",
"attributes": {
"configuration_key": "height",
"configuration_value": 1024,
"diffusion_model_name": "SDXL"
}
},
{
"type": "DiffusionModelDefaultConfiguration",
"attributes": {
"configuration_key": "width",
"configuration_value": 1024,
"diffusion_model_name": "SDXL"
}
}
]
}
}
]
}
POST /api/models
$ curl -X POST -d '{"name": "My Model", "checkpoint": "sd_xl_base_1.0.safetensors"}' https://app.enfugue.ai:45554/api/models
{
"meta": {
"params": {}
},
"data": {
"type": "DiffusionModel",
"attributes": {
"model": "sd_xl_base_1.0.safetensors",
"name": "My Model",
"negative_prompt": "",
"prompt": "",
"size": 512
}
}
}
PATCH /api/models/{{ name }}
$ curl -X PATCH -d '{"scheduler": "ddim"}' https://app.enfugue.ai:45554/api/models/My%20Model
{
"meta": {
"params": {}
},
"data": {
"type": "DiffusionModel",
"attributes": {
"model": "sd_xl_base_1.0.safetensors",
"name": "My Model",
"negative_prompt": "",
"prompt": "",
"size": 512
}
}
}
PATCH /api/models/{{ name }}
$ curl -X DELETE https://app.enfugue.ai:45554/api/models/My%20Model
{
"meta": {
"params": {}
},
"data": null
}
GET /api/invocation-history
$ curl https://app.enfugue.ai:45554/api/invocation-history?limit=1
{
"meta": {
"params": {
"limit": [
"1"
]
},
"count": 77
},
"data": [
{
"type": "DiffusionInvocation",
"attributes": {
"duration": 339.707587,
"error": null,
"id": "cf167ce51bd8433b96d46774fd5a61e9",
"outputs": 1,
"plan": {
"model": "C:\\Users\\painebenjamin\\.cache\\enfugue\\checkpoint\\sd_xl_base_1.0.safetensors",
"refiner": "C:\\Users\\painebenjamin\\.cache\\enfugue\\checkpoint\\sd_xl_refiner_1.0.safetensors",
"inpainter": "C:\\Users\\painebenjamin\\.cache\\enfugue\\checkpoint\\realisticVisionV40_v40VAE-inpainting.safetensors",
"lora": [],
"lycoris": [],
"inversion": [],
"scheduler": null,
"multi_scheduler": null,
"vae": null,
"width": 1024,
"height": 1024,
"size": 1024,
"inpainter_size": 512,
"refiner_size": 1024,
"seed": null,
"prompt": "A happy-looking puppy",
"negative_prompt": "",
"image": null,
"image_callback_steps": 10,
"nodes": [
{
"width": 1024,
"height": 1024,
"prompt": "A happy-looking puppy,RAW photo,highly detailed,intricate detail,4k,8k,sharp focus,ultra-detailed,portrait photography,cinestill 800t,Fujifilm XT3",
"negative_prompt": "",
"controlnet": null,
"conditioning_scale": 1,
"strength": 0.8,
"num_inference_steps": 50,
"guidance_scale": 7.5,
"remove_background": false,
"refiner_strength": 0.3,
"refiner_guidance_scale": 5,
"refiner_aesthetic_score": 6,
"refiner_negative_aesthetic_score": 2.5,
"process_control_image": true,
"scale_to_model_size": true,
"image": null,
"mask": null,
"control_image": null,
"children": [],
"bounds": [
[
0,
0
],
[
1024,
1024
]
]
}
],
"samples": 1,
"upscale": {
"method": [
"esrgan"
],
"amount": 2,
"iterative": true,
"diffusion": {
"steps": [
100
],
"guidance_scale": [
12
],
"chunking_size": 256,
"chunking_blur": 256,
"strength": [
0.2
],
"prompt": [
"A happy-looking puppy,RAW photo,highly detailed,intricate detail,death and decay incarnated as a fetish of sin,4k,8k,sharp focus,ultra-detailed,portrait photography,cinestill 800t,Fujifilm XT3"
],
"negative_prompt": [
""
],
"controlnet": [],
"scale_chunking_size": true,
"scale_chunking_blur": true
}
},
"chunking_size": 64,
"chunking_blur": 64,
"build_tensorrt": false
},
"started": "2023-07-27T16:34:12",
"user_id": 0
}
}
]
}
DELETE /api/invocation/{{ uuid }}
$ curl -X DELETE https://app.enfugue.ai:45554/api/invocation/4547f0f712084414940006bc1fbfb8b2
{
"meta": {
"params": {}
},
"data": null
}
These endpoints still use JSON, but are not performing representative state transfer.
GET /api/checkpoints
$ curl https://app.enfugue.ai:45554/api/checkpoints
{
"meta": {
"params": {}
},
"data": [
"cyberrealistic_v30-inpainting.safetensors",
"cyberrealistic_v30.safetensors",
"darkSushiMixMix_225D.safetensors",
"deliberate_v2.ckpt",
"dreamshaperXL10_alpha2Xl10.safetensors",
"dreamshaper_6BakedVae-inpainting.safetensors",
"dreamshaper_6BakedVae.safetensors",
"dreamshaper_7-inpainting.safetensors",
"dreamshaper_7.safetensors",
"icbinpICantBelieveIts_v8.ckpt",
"meinamix_meinaV10.safetensors",
"meinamix_meinaV11.safetensors",
"realisticVisionV20_v20-inpainting.ckpt",
"realisticVisionV20_v20.ckpt",
"realisticVisionV30_v30VAE.safetensors",
"realisticVisionV40_v40VAE-inpainting.safetensors",
"realisticVisionV40_v40VAE.safetensors",
"reliberate_v10.safetensors",
"sd-v1-5-inpainting.ckpt",
"sd_xl_base_1.0.safetensors",
"sd_xl_refiner_1.0.safetensors",
"taurealMix_v37Fp16prunedVae.safetensors",
"v1-5-pruned.ckpt"
]
}
GET /api/lora
$ curl https://app.enfugue.ai:45554/api/lora
{
"meta": {
"params": {}
},
"data": [
"3DMM_V12.safetensors",
"margot_lora_sdxl_v1-000006.safetensors",
"pixel-art-xl.safetensors",
"sd_xl_offset_example-lora_1.0.safetensors"
]
}
GET /api/lycoris
$ curl https://app.enfugue.ai:45554/api/lycoris
{
"meta": {
"params": {}
},
"data": [
"SM(120R).safetensors"
]
}
GET /api/inversion
$ curl https://app.enfugue.ai:45554/api/inversions
{
"meta": {
"params": {}
},
"data": [
"EasyNegativeV2.safetensors"
]
}
GET /api/settings
$ curl https://app.enfugue.ai:45554/api/settings
{
"meta": {
"params": {}
},
"data": {
"safe": false,
"auth": false,
"max_queued_invocations": 4,
"max_queued_downloads": 10,
"max_concurrent_downloads": 2,
"switch_mode": "offload",
"cache_mode": "xl",
"precision": null
}
}
POST /api/settings/
$ curl -X POST -d '{"cache_mode": null}' https://app.enfugue.ai:45554/api/settings
{
"meta": {
"params": {}
},
"data": {
"safe": false,
"auth": false,
"max_queued_invocations": 4,
"max_queued_downloads": 10,
"max_concurrent_downloads": 2,
"switch_mode": "offload",
"cache_mode": null,
"precision": null
}
}
POST /api/invoke
There are many parameters to this endpoint, which control how Stable Diffusion will be instantiated and invoked. The parameters are:
-
model
- Either a filename for a checkpoint or a model name. The filename can be an absolute path, or one relative to the configured checkpoint directory. -
model_type
- Optional, forcecheckpoint
ormodel
lookup in the case of collision. -
chunking_size
- The number of pixels to move the frame when doing chunked diffusion. When this number is greater than 0, the engine will only ever process a square in the size of the configured model size at once. After each square, the frame will be moved by this many pixels along either the horizontal or vertical axis, and then the image is re-diffused. When this number is 0, chunking is disabled, and the entire canvas will be diffused at once. Disabling this (setting it to 0) can have varying visual results, but a guaranteed result is drastically increased VRAM usage for large images. A low number can produce more detailed results, but can be noisy, and takes longer to process. A high number is faster to process, but can have poor results especially along frame boundaries. -
chunking_blur
- The number of pixels to feather along the edge of the frame when blending chunked diffusions together. Low numbers can produce less blurry but more noisy results, and can potentially result in visible breaks in the frame. High numbers can help blend frames, but produce blurrier results. -
size
- The size of the diffusion engine when using multi-diffusion. -
refiner
- When not using preconfigured models, you can pass a checkpoint to user as a refiner. By default no refiner is used. -
refiner_size
- The size of the engine when using the refiner. -
inpainter
- When not using preconfigured models, you can pass a checkpoint to additionally use as an inpainter. By default, the primary model will be converted for inpainting. -
inpainter_size
- The size of the engine when using the inpainter. -
lora
- When not using preconfigured models, you can load one or more LoRA. Pass a single string to load a single model with full weight. Pass a dictionary withmodel
andweight
keys to load a model with a certain weight. Pass an array of either of these to load multiple. -
lycoris
- The same as LoRA, but for LyCORIS. -
inversion
- The same as LoRA, but for textual inversion. Inversion does not have a weight parameter. -
vae
- The name of a VAE to load instead of default. Options areema|mse|xl|xl16
.
-
seed
- A seed to use instead of randomized. -
width
andheight
- The dimensions of the image to create. Default matches the engine size. -
prompt
andnegative_prompt
- These are global and apply to the main diffusion task and all child tasks -
image
- Pass the path to an image or a Data URI for img2img inference or inpainting. -
strength
- When doingimg2img
(image without mask), this is how much of the image to denoise, with 0.0 being none, and 1.0 being total destruction. Default value is 0.8. -
mask
- When combined with animage
, this is the inpainting mask. -
invert_mask
- If your mask needs to be inverted (i.e. you have white for the areas you want painted instead of black), pass this astrue
to invert prior to use, -
fit
- When using an image and also passing width/height, control how the image is fit within the size. Options areactual|cover|contain|stretch
-
anchor
- When using an image and also passing width/height, control how the image is fit within the frame. Options aretop-left|top-center|top-right|center-left|center-center|center-right|bottom-left|bottom-center|bottom-right
-
control_image
- Use this image for a control image. -
controlnet
- The controlnet to use when paired with an image. Options arecanny|mlsd|hed|scribble|tile|inpaint|depth|normal|pose|pidi|line|anime
-
conditioning_scale
- The weight of the controlnet, from 0.0 to 1.0. -
process_control_image
- Whether or not to run the control image through the appropriate transformation algorithm for the controlnet. Default true. -
invert_control_image
- When not usingprocess_control_image
, you should pass a black and white conditioning image. If your image is black on white background, passinvert_control_image
to invert it prior to its use. -
remove_background
- Remove either the input image's background or the background of the result image. -
fill_background
- When paired withremove_background
and an image, this will remove the background and then fill it with diffusion. -
scale_to_model_size
- When using a small image or wanting to generate an image smaller than the size of the engine, it can be scaled up so it's smallest dimension is the size of the image. This can produce better images, but takes longer. Default false. -
samples
- The number of samples to generate at a time -
num_inference_steps
- How many steps to take during primary inference, larger values take longer to process but can produce better results. Default is 40. -
guidance_scale
- How closely to follow the text prompt; high values result in high-contrast images closely adhering to your text, low values result in low-contrast images with more randomness. Default is 7. -
scheduler
andmulti_scheduler
- The scheduler to use during regular and multi-diffusion. Options areddim|ddpm|deis|dpmsm|dpmss|heun|dpmd|adpmd|dpmsde|unipc|lmsd|pndm|eds|eads
-
refiner_strength
- When using a refiner, this will control how much of the original image is kept, and how much of it is replaced with refined content. A value of 1.0 represents total destruction of the first image. A value of 0 will not perform any refinement. Default is 0.3. -
refiner_aesthetic_score
- Aesthetic scores are assigned to images in SDXL refinement; this controls the positive score. Default is 6.0. -
refiner_negative_aesthetic_score
- The same as above, but for the negative. Default is 2.5. -
refiner_guidance_scale
- The same asguidance_scale
, but for when using the refiner. Default is 5.0.
-
outscale
- Scale up to this amount using upscaling. You should pass 2, 4, 8, or 16. -
upscale
- The method of upscaling to use. Either a single value or an array of values for iterative upscaling. Options areesrgan|esrganime|gfpgan|lanczos|bilinear|bicubic|nearest
. -
upscale_iterative
- Whether or not to upscale iteratively. When passed astrue
, the image will be re-doubled until it reaches your target scale, instead of directly jumping to 4, 8 or 16. -
upscale_diffusion
- Whether or not to re-diffuse upscaled samples. When passed astrue
, the image will be re-ran through the diffusion engine. This is similar to the "Ultimate SD Upscale" method. -
upscale_diffusion_steps
- The number of inference steps to use when upscaling. Default is 100. Can be an array for each step of an iterative upscale. -
upscale_diffusion_guidance_scale
- The guidance scale to use when upscaling. Default is 12. Can be an array for each step of an iterative upscale. -
upscale_diffusion_strength
- The denoising strength to use when upscaling. Default is 0.2. Can be an array for each step of an iterative upscale. -
upscale_diffusion_prompt
- A prompt to use when doing upscale diffusion. Default is the same as the global prompt. Can be an array for each step of an iterative upscale. -
upscale_diffusion_negative_prompt
- A negative prompt to use when doing upscale diffusion. Default is the same as the global negative prompt. Can be an array for each step of an iterative upscale. -
upscale_diffusion_controlnet
- An optional controlnet to use when doing upscale diffusion (liketile
.) Can be an array for each step of an iterative upscale. -
upscale_diffusion_chunking_size
- An optional chunking size when doing upscaling. In general, you'd want to make this higher than the base chunking size, in order to cut down on necessary iterations when upscaling. -
upscale_diffusion_chunking_blur
- Similar to the global chunking blur but for the chunking size. -
upscale_diffusion_scale_chunking_size
- Whether or not to double the chunking size with each iteration. It will never become larger than half the engine size. Default true. -
upscale_diffusion_scale_chunking_size
- Whether or not to double the chunking blur with each iteration. It will never become larger than half the engine size. Default true.
$ curl -d '{"prompt": "Hello, world!"}' https://app.enfugue.ai:45554/api/invoke
{
"meta": {
"params": {}
},
"data": {
"id": 1,
"uuid": "fb8cb1cede594ef2b7f5171bc5fc47dd",
"status": "processing",
"progress": null,
"step": null,
"duration": 3.089267,
"total": null,
"images": null,
"rate": null
}
}
You can use the nodes
parameter to isolate your invocation parameters to a certain area of the canvas; this is how the UI builds and executes it's plan for outpainting, region prompts, etc. The nodes
parameter takes the majority of the same arguments above, and additionally accepts four more parameters: x
, y
, w
and h
. These correspond to the pixel positions and dimensions of the node on the canvas.
For example, this invocation payload will create a cat on the left half of an image, and a dog on the right half. It will remove the backgrounds surrounding each subject, then fill in the space around both in a total of three steps.
{
"width": 512,
"height": 512,
"prompt": "A cat and a dog sleeping peacefully on a rug",
"nodes": [
{
"x": 0,
"y": 0,
"w": 256,
"h": 512,
"prompt": "A cat sleeping peacefully on a rug",
"remove_background": true
},
{
"x": 256,
"y": 0,
"w": 256,
"h": 512,
"prompt": "A dog sleeping peacefully on a rug",
"remove_background": true
}
]
}
GET /api/invocation/{{ uuid }}
Status is one of queued|processing|completed|error
$ curl https://app.enfugue.ai:45554/api/invocation/fb8cb1cede594ef2b7f5171bc5fc47dd
{
"meta": {
"params": {}
},
"data": {
"id": 1,
"uuid": "fb8cb1cede594ef2b7f5171bc5fc47dd",
"status": "completed",
"progress": 1,
"step": 45,
"duration": 31.177126,
"total": 45,
"images": [
"images/fb8cb1cede594ef2b7f5171bc5fc47dd_0.png"
],
"rate": 12.343350538318203
}
}
GET /api/invocation/images/{{ image }}
$ curl https://app.enfugue.ai:45554/api/invocation/images/fb8cb1cede594ef2b7f5171bc5fc47dd_0.png > myfile.png
$ identify myfile.png
myfile.png PNG 512x512 512x512+0+0 8-bit sRGB 417065B 0.000u 0:00.001
// TODO
// TODO