data_types

Comfy types and python objects

Comfy type	Python	Notes
CLIP	comfy.sd.CLIP	See below
CONDITIONING	a list of conditionings	see below
FLOAT	float	As an input you can specify "min", "max", "default", "step"
IMAGE	torch.Tensor with shape [B,H,W,C]	A batch of B images. C=3 (RGB)
INT	int	As an input you can specify "min", "max", "default"
LATENT	dict	'samples' is the key for the samples as a torch.Tensor
MASK	torch.Tensor with shape [H,W] or [B,C,H,W]	Note the shape difference from IMAGE!
MODEL	model_base.BaseModel or model_base.SD21UNCLIP	see model_base.py
STRING	str	As an input type a "default" must be provided
VAE	comfy.sd.VAE	See below

Conditioning

The object passed is a list, each element of which represents a conditioning (so the list can represent one or many conditionings). Each entry in the list is itself a list of length 2, containing a tensor (shape, at least for SDXL, [1,77,2048]) which holds the 77 conditioning vectors (I assume the first dimension is batch?), and a dictionary which initially holds the key pooled_output with, as a value, a tensor (shape [1,1280]). Other things can be added to this dictionary, to control the use of the conditioning; see the various ConditioningSet.... nodes.

comfy.sd.CLIP

See the definition in sd.py, and especially the way it is constructed in the load_checkpoint method.

A CLIP object holds:

• a comfy.model_patcher.ModelPatcher that wraps a model and allows patches to be applied to it (see below)
• the underlying clip which is generally either sd2_clip.SD2ClipModel or sd1_clip.SD1ClipModel
• a tokenizer, which is generally either sd2_clip.SD2Tokenizer or sd1_clip.SD1Tokenizer
• a layer_idx (used for clip_skip?)

comfy.sd.VAE

A VAE is basically a wrapper for its self.first_stage_model, which is an ldm.models.autoencoder.AutoencoderKL

Patches

Models in Comfy are (can be) wrapped in a ModelPatcher class which allows patches (such as LoRAs) to be applied to them; the ModelPatcher basically just holds the patches until the model is run, and then they are applied to the appropriate steps of the model.