Package org.freedesktop.gstreamer.video

Class VideoEncoder

java.lang.Object

org.javagi.base.ProxyInstance

org.gnome.gobject.TypeInstance

org.gnome.gobject.GObject

org.gnome.gobject.InitiallyUnowned

org.freedesktop.gstreamer.gst.GstObject

org.freedesktop.gstreamer.gst.Element

org.freedesktop.gstreamer.video.VideoEncoder

All Implemented Interfaces:

Preset, Proxy

Direct Known Subclasses:

VideoEncoder.VideoEncoder$Impl

@Generated("io.github.jwharm.JavaGI")
public abstract class VideoEncoder
extends Element
implements Preset

This base class is for video encoders turning raw video into encoded video data.

GstVideoEncoder and subclass should cooperate as follows.

Configuration

• Initially, GstVideoEncoder calls start when the encoder element is activated, which allows subclass to perform any global setup.
• GstVideoEncoder calls setFormat to inform subclass of the format of input video data that it is about to receive. Subclass should setup for encoding and configure base class as appropriate (e.g. latency). While unlikely, it might be called more than once, if changing input parameters require reconfiguration. Baseclass will ensure that processing of current configuration is finished.
• GstVideoEncoder calls stop at end of all processing.

Data processing

• Base class collects input data and metadata into a frame and hands this to subclass' handleFrame.

• If codec processing results in encoded data, subclass should call gstVideoEncoderFinishFrame to have encoded data pushed downstream.

• If implemented, baseclass calls subclass prePush just prior to pushing to allow subclasses to modify some metadata on the buffer. If it returns GST_FLOW_OK, the buffer is pushed downstream.

• GstVideoEncoderClass will handle both srcpad and sinkpad events. Sink events will be passed to subclass if event callback has been provided.

Shutdown phase

• GstVideoEncoder class calls stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It should also be able to provide fixed src pad caps in getcaps by the time it calls gstVideoEncoderFinishFrame.

Things that subclass need to take care of:

• Provide pad templates
• Provide source pad caps before pushing the first buffer
• Accept data in handleFrame and provide encoded results to gstVideoEncoderFinishFrame.

The GstVideoEncoder:qos property will enable the Quality-of-Service features of the encoder which gather statistics about the real-time performance of the downstream elements. If enabled, subclasses can use gst_video_encoder_get_max_encode_time() to check if input frames are already late and drop them right away to give a chance to the pipeline to catch up.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	VideoEncoder.Builder<B extends VideoEncoder.Builder<B>>	Inner class implementing a builder pattern to construct a GObject with properties.
static class	VideoEncoder.VideoEncoder$Impl	The VideoEncoder$Impl type represents a native instance of the abstract VideoEncoder class.
static class	VideoEncoder.VideoEncoderClass	Subclasses can override any of the available virtual methods or not, as needed.

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.Element

Element.Element$Impl, Element.ElementClass, Element.NoMorePadsCallback, Element.PadAddedCallback, Element.PadRemovedCallback

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.GstObject

GstObject.DeepNotifyCallback, GstObject.Object$Impl, GstObject.ObjectClass

Nested classes/interfaces inherited from class org.gnome.gobject.InitiallyUnowned

InitiallyUnowned.InitiallyUnownedClass

Nested classes/interfaces inherited from class org.gnome.gobject.GObject

GObject.NotifyCallback

Nested classes/interfaces inherited from interface org.freedesktop.gstreamer.gst.Preset

Preset.Preset$Impl, Preset.PresetInterface

Constructor Summary

Constructors

Constructor	Description
VideoEncoder()	Creates a new VideoEncoder.
VideoEncoder(MemorySegment address)	Create a VideoEncoder proxy instance for the provided memory address.

Method Summary

Modifier and Type	Method	Description
Buffer	allocateOutputBuffer(long size)	Helper function that allocates a buffer to hold an encoded video frame for encoder's current GstVideoCodecState.
FlowReturn	allocateOutputFrame(VideoCodecFrame frame, long size)	Helper function that allocates a buffer to hold an encoded video frame for encoder's current GstVideoCodecState.
protected VideoEncoder	asParent()	Returns this instance as if it were its parent type.
protected boolean	close()	Optional.
protected boolean	decideAllocation(Query query)	Optional.
void	dropFrame(VideoCodecFrame frame)	Removes frame from the list of pending frames, releases it and posts a QoS message with the frame's details on the bus.
protected FlowReturn	finish()	Optional.
FlowReturn	finishFrame(VideoCodecFrame frame)	frame must have a valid encoded data buffer, whose metadata fields are then appropriately set according to frame data or no buffer at all if the frame should be dropped.
FlowReturn	finishSubframe(VideoCodecFrame frame)	If multiple subframes are produced for one input frame then use this method for each subframe, except for the last one.
protected boolean	flush()	Optional.
void	getAllocator(@Nullable Out<Allocator> allocator, @Nullable AllocationParams params)	Lets GstVideoEncoder sub-classes to know the memory allocator used by the base class and its params.
protected Caps	getcaps(Caps filter)	Optional.
@Nullable VideoCodecFrame	getFrame(int frameNumber)	Get a pending unfinished GstVideoCodecFrame
List<VideoCodecFrame>	getFrames()	Get all pending unfinished GstVideoCodecFrame
void	getLatency(@Nullable ClockTime minLatency, @Nullable ClockTime maxLatency)	Query the configured encoding latency.
ClockTimeDiff	getMaxEncodeTime(VideoCodecFrame frame)	Determines maximum possible encoding time for frame that will allow it to encode and arrive in time (as determined by QoS events).
static MemoryLayout	getMemoryLayout()	The memory layout of the native struct.
ClockTime	getMinForceKeyUnitInterval()	Returns the minimum force-keyunit interval, see gst_video_encoder_set_min_force_key_unit_interval() for more details.
@Nullable VideoCodecFrame	getOldestFrame()	Get the oldest unfinished pending GstVideoCodecFrame
@Nullable VideoCodecState	getOutputState()	Get the current GstVideoCodecState
static @Nullable Type	getType()	Get the GType of the VideoEncoder class
protected FlowReturn	handleFrame(VideoCodecFrame frame)	Provides input frame to subclass.
boolean	isQosEnabled()	Checks if this VideoEncoder is currently configured to handle Quality-of-Service events from downstream.
void	mergeTags(@Nullable TagList tags, TagMergeMode mode)	Sets the video encoder tags and how they should be merged with any upstream stream tags.
boolean	negotiate()	Negotiate with downstream elements to currently configured GstVideoCodecState.
protected boolean	open()	Optional.
protected FlowReturn	prePush(VideoCodecFrame frame)	Optional.
protected boolean	proposeAllocation(Query query)	Optional.
Caps	proxyGetcaps(@Nullable Caps caps, @Nullable Caps filter)	Returns caps that express caps (or sink template caps if caps == NULL) restricted to resolution/format/...
void	releaseFrame(VideoCodecFrame frame)	Removes frame from list of pending frames and releases it, similar to calling gst_video_encoder_finish_frame() without a buffer attached to the frame, but does not post a QoS message or do any additional processing.
protected boolean	reset(boolean hard)	Optional.
protected boolean	setFormat(VideoCodecState state)	Optional.
void	setHeaders(List<Buffer> headers)	Set the codec headers to be sent downstream whenever requested.
void	setLatency(ClockTime minLatency, ClockTime maxLatency)	Informs baseclass of encoding latency.
void	setMinForceKeyUnitInterval(ClockTime interval)	Sets the minimum interval for requesting keyframes based on force-keyunit events.
void	setMinPts(ClockTime minPts)	Request minimal value for PTS passed to handle_frame.
@Nullable VideoCodecState	setOutputState(Caps caps, @Nullable VideoCodecState reference)	Creates a new GstVideoCodecState with the specified caps as the output state for the encoder.
void	setQosEnabled(boolean enabled)	Configures this VideoEncoder to handle Quality-of-Service events from downstream.
protected boolean	sinkEvent(Event event)	Optional.
protected boolean	sinkQuery(Query query)	Optional.
protected boolean	srcEvent(Event event)	Optional.
protected boolean	srcQuery(Query query)	Optional.
protected boolean	start()	Optional.
protected boolean	stop()	Optional.
protected boolean	transformMeta(VideoCodecFrame frame, Meta meta)	Optional.

Methods inherited from class org.freedesktop.gstreamer.gst.Element

abortState, addPad, addPropertyDeepNotifyWatch, addPropertyNotifyWatch, callAsync, changeState, continueState, createAllPads, decorateStreamId, decorateStreamIdPrintf, emitNoMorePads, emitPadAdded, emitPadRemoved, foreachPad, foreachSinkPad, foreachSrcPad, getBaseTime, getBus, getClock, getCompatiblePad, getCompatiblePadTemplate, getContext, getContexts, getContextUnlocked, getCurrentClockTime, getCurrentRunningTime, getFactory, getMetadata, getPadTemplate, getPadTemplateList, getRequestPad, getStartTime, getState, getStaticPad, isLockedState, iteratePads, iterateSinkPads, iterateSrcPads, link, linkFiltered, linkMany, linkPads, linkPadsFiltered, linkPadsFull, linkPadsFull, lostState, makeFromUri, messageFull, messageFull, messageFullWithDetails, messageFullWithDetails, noMorePads, onNoMorePads, onPadAdded, onPadRemoved, padAdded, padRemoved, postMessage, provideClock, query, queryConvert, queryDuration, queryPosition, register, releasePad, releaseRequestPad, removePad, removePropertyNotifyWatch, requestPad, requestPadSimple, seek, seek, seekSimple, seekSimple, sendEvent, setBaseTime, setBus, setClock, setContext, setLockedState, setStartTime, setState, stateChanged, stateChangeReturnGetName, stateGetName, syncStateWithParent, typeSetSkipDocumentation, unlink, unlinkMany, unlinkPads

Methods inherited from class org.freedesktop.gstreamer.gst.GstObject

addControlBinding, checkUniqueness, deepNotify, defaultDeepNotify, defaultError, emitDeepNotify, getControlBinding, getControlRate, getGValueArray, getName, getParent, getPathString, getValue, getValueArray, hasActiveControlBindings, hasAncestor, hasAsAncestor, hasAsParent, onDeepNotify, ref, refSink, removeControlBinding, replace, setControlBindingDisabled, setControlBindingsDisabled, setControlRate, setName, setParent, suggestNextSync, syncValues, unparent, unref

Methods inherited from class org.gnome.gobject.InitiallyUnowned

builder

Methods inherited from class org.gnome.gobject.GObject

addToggleRef, addWeakPointer, bindProperty, bindProperty, bindProperty, bindPropertyFull, bindPropertyFull, bindPropertyWithClosures, bindPropertyWithClosures, compatControl, connect, connect, connect, constructed, disconnect, dispatchPropertiesChanged, dispose, dupData, dupQdata, emit, emitNotify, finalize_, forceFloating, freezeNotify, get, getData, getProperty, getProperty, getProperty, getQdata, getv, interfaceFindProperty, interfaceInstallProperty, interfaceListProperties, isFloating, newInstance, newInstance, newv, notify, notify, notifyByPspec, onNotify, refSink, removeToggleRef, removeWeakPointer, replaceData, replaceQdata, runDispose, set, setData, setDataFull, setProperty, setProperty, setProperty, setQdata, setQdataFull, setv, stealData, stealQdata, takeRef, thawNotify, watchClosure, weakRef, weakUnref, withProperties

Methods inherited from class org.gnome.gobject.TypeInstance

callParent, callParent, cast, getPrivate, readGClass, writeGClass

Methods inherited from class org.javagi.base.ProxyInstance

equals, handle, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.freedesktop.gstreamer.gst.Preset

deletePreset, getMeta, getPresetNames, getPropertyNames, isEditable, loadPreset, renamePreset, savePreset, setMeta

Methods inherited from interface org.javagi.base.Proxy

handle

Constructor Details

VideoEncoder

public VideoEncoder(MemorySegment address)

Create a VideoEncoder proxy instance for the provided memory address.

Parameters:

address - the memory address of the native object

VideoEncoder

public VideoEncoder()

Creates a new VideoEncoder.

Method Details

getType

public static @Nullable Type getType()

Get the GType of the VideoEncoder class

Returns:

the GType

getMemoryLayout

public static MemoryLayout getMemoryLayout()

The memory layout of the native struct.

Returns:

the memory layout

asParent

protected VideoEncoder asParent()

Returns this instance as if it were its parent type. This is mostly synonymous to the Java super keyword, but will set the native typeclass function pointers to the parent type. When overriding a native virtual method in Java, "chaining up" with super.methodName() doesn't work, because it invokes the overridden function pointer again. To chain up, call asParent().methodName(). This will call the native function pointer of this virtual method in the typeclass of the parent type.

Overrides:

asParent in class Element

allocateOutputBuffer

public Buffer allocateOutputBuffer(long size)

Helper function that allocates a buffer to hold an encoded video frame for encoder's current GstVideoCodecState.

Parameters:

size - size of the buffer

Returns:

allocated buffer

allocateOutputFrame

public FlowReturn allocateOutputFrame(VideoCodecFrame frame,
 long size)

Helper function that allocates a buffer to hold an encoded video frame for encoder's current GstVideoCodecState. Subclass should already have configured video state and set src pad caps.

The buffer allocated here is owned by the frame and you should only keep references to the frame, not the buffer.

Parameters:

frame - a GstVideoCodecFrame

size - size of the buffer

Returns:

FlowReturn.OK if an output buffer could be allocated

dropFrame

public void dropFrame(VideoCodecFrame frame)

Removes frame from the list of pending frames, releases it and posts a QoS message with the frame's details on the bus. Similar to calling gst_video_encoder_finish_frame() without a buffer attached to frame, but this function additionally stores events from frame as pending, to be pushed out alongside the next frame submitted via gst_video_encoder_finish_frame().

Parameters:

frame - a GstVideoCodecFrame

Since:

1.26

finishFrame

public FlowReturn finishFrame(VideoCodecFrame frame)

frame must have a valid encoded data buffer, whose metadata fields are then appropriately set according to frame data or no buffer at all if the frame should be dropped. It is subsequently pushed downstream or provided to prePush. In any case, the frame is considered finished and released.

If frame does not have a buffer attached, it will be dropped, and a QoS message will be posted on the bus. Events from frame will be pushed out immediately.

After calling this function the output buffer of the frame is to be considered read-only. This function will also change the metadata of the buffer.

Parameters:

frame - an encoded GstVideoCodecFrame

Returns:

a GstFlowReturn resulting from sending data downstream

finishSubframe

public FlowReturn finishSubframe(VideoCodecFrame frame)

If multiple subframes are produced for one input frame then use this method for each subframe, except for the last one. Before calling this function, you need to fill frame->output_buffer with the encoded buffer to push.

You must call gst_video_encoder_finish_frame() for the last sub-frame to tell the encoder that the frame has been fully encoded.

This function will change the metadata of frame and frame->output_buffer will be pushed downstream.

Parameters:

frame - a GstVideoCodecFrame being encoded

Returns:

a GstFlowReturn resulting from pushing the buffer downstream.

Since:

1.18

getAllocator

public void getAllocator(@Nullable Out<Allocator> allocator,
 @Nullable AllocationParams params)

Lets GstVideoEncoder sub-classes to know the memory allocator used by the base class and its params.

Unref the allocator after use it.

Parameters:

allocator - the GstAllocator used

params - the GstAllocationParams of allocator

getFrame

public @Nullable VideoCodecFrame getFrame(int frameNumber)

Get a pending unfinished GstVideoCodecFrame

Parameters:

frameNumber - system_frame_number of a frame

Returns:

pending unfinished GstVideoCodecFrame identified by frameNumber.

getFrames

public List<VideoCodecFrame> getFrames()

Get all pending unfinished GstVideoCodecFrame

Returns:

pending unfinished GstVideoCodecFrame.

getLatency

public void getLatency(@Nullable ClockTime minLatency,
 @Nullable ClockTime maxLatency)

Query the configured encoding latency. Results will be returned via minLatency and maxLatency.

Parameters:

minLatency - address of variable in which to store the configured minimum latency, or null

maxLatency - address of variable in which to store the configured maximum latency, or null

getMaxEncodeTime

public ClockTimeDiff getMaxEncodeTime(VideoCodecFrame frame)

Determines maximum possible encoding time for frame that will allow it to encode and arrive in time (as determined by QoS events). In particular, a negative result means encoding in time is no longer possible and should therefore occur as soon/skippy as possible.

If no QoS events have been received from downstream, or if GstVideoEncoder:qos is disabled this function returns G_MAXINT64.

Parameters:

frame - a GstVideoCodecFrame

Returns:

max decoding time.

Since:

1.14

getMinForceKeyUnitInterval

public ClockTime getMinForceKeyUnitInterval()

Returns the minimum force-keyunit interval, see gst_video_encoder_set_min_force_key_unit_interval() for more details.

Returns:

the minimum force-keyunit interval

Since:

1.18

getOldestFrame

public @Nullable VideoCodecFrame getOldestFrame()

Get the oldest unfinished pending GstVideoCodecFrame

Returns:

oldest unfinished pending GstVideoCodecFrame

getOutputState

public @Nullable VideoCodecState getOutputState()

Get the current GstVideoCodecState

Returns:

GstVideoCodecState describing format of video data.

isQosEnabled

public boolean isQosEnabled()

Checks if this VideoEncoder is currently configured to handle Quality-of-Service events from downstream.

Returns:

true if the encoder is configured to perform Quality-of-Service.

Since:

1.14

mergeTags

public void mergeTags(@Nullable TagList tags,
 TagMergeMode mode)

Sets the video encoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_video_encoder_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

MT safe.

Parameters:

tags - a GstTagList to merge, or NULL to unset previously-set tags

mode - the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE

negotiate

public boolean negotiate()

Negotiate with downstream elements to currently configured GstVideoCodecState. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

Returns:

true if the negotiation succeeded, else false.

proxyGetcaps

public Caps proxyGetcaps(@Nullable Caps caps,
 @Nullable Caps filter)

Returns caps that express caps (or sink template caps if caps == NULL) restricted to resolution/format/... combinations supported by downstream elements (e.g. muxers).

Parameters:

caps - initial caps

filter - filter caps

Returns:

a GstCaps owned by caller

releaseFrame

public void releaseFrame(VideoCodecFrame frame)

Removes frame from list of pending frames and releases it, similar to calling gst_video_encoder_finish_frame() without a buffer attached to the frame, but does not post a QoS message or do any additional processing. Events from frame are moved to the pending events list.

Parameters:

frame - a GstVideoCodecFrame

Since:

1.26

setHeaders

public void setHeaders(List<Buffer> headers)

Set the codec headers to be sent downstream whenever requested.

Parameters:

headers - a list of GstBuffer containing the codec header

setLatency

public void setLatency(ClockTime minLatency,
 ClockTime maxLatency)

Informs baseclass of encoding latency. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

Parameters:

minLatency - minimum latency

maxLatency - maximum latency

setMinForceKeyUnitInterval

public void setMinForceKeyUnitInterval(ClockTime interval)

Sets the minimum interval for requesting keyframes based on force-keyunit events. Setting this to 0 will allow to handle every event, setting this to GST_CLOCK_TIME_NONE causes force-keyunit events to be ignored.

Parameters:

interval - minimum interval

Since:

1.18

setMinPts

public void setMinPts(ClockTime minPts)

Request minimal value for PTS passed to handle_frame.

For streams with reordered frames this can be used to ensure that there is enough time to accommodate first DTS, which may be less than first PTS

Parameters:

minPts - minimal PTS that will be passed to handle_frame

Since:

1.6

setOutputState

public @Nullable VideoCodecState setOutputState(Caps caps,
 @Nullable VideoCodecState reference)

Creates a new GstVideoCodecState with the specified caps as the output state for the encoder. Any previously set output state on this VideoEncoder will be replaced by the newly created one.

The specified caps should not contain any resolution, pixel-aspect-ratio, framerate, codec-data, .... Those should be specified instead in the returned GstVideoCodecState.

If the subclass wishes to copy over existing fields (like pixel aspect ratio, or framerate) from an existing GstVideoCodecState, it can be provided as a reference.

If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned GstVideoCodecState.

The new output state will only take effect (set on pads and buffers) starting from the next call to gst_video_encoder_finish_frame().

Parameters:

caps - the GstCaps to use for the output

reference - An optional reference GstVideoCodecState

Returns:

the newly configured output state.

setQosEnabled

public void setQosEnabled(boolean enabled)

Configures this VideoEncoder to handle Quality-of-Service events from downstream.

Parameters:

enabled - the new qos value.

Since:

1.14

close

protected boolean close()

Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources.

decideAllocation

protected boolean decideAllocation(Query query)

Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query. Subclasses should chain up to the parent implementation to invoke the default handler.

finish

protected FlowReturn finish()

Optional. Called to request subclass to dispatch any pending remaining data (e.g. at EOS).

flush

protected boolean flush()

Optional. Flush all remaining data from the encoder without pushing it downstream. Since: 1.2

getcaps

protected Caps getcaps(Caps filter)

Optional. Allows for a custom sink getcaps implementation (e.g. for multichannel input specification). If not implemented, default returns gst_video_encoder_proxy_getcaps applied to sink template caps.

handleFrame

protected FlowReturn handleFrame(VideoCodecFrame frame)

Provides input frame to subclass.

open

protected boolean open()

Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources.

prePush

protected FlowReturn prePush(VideoCodecFrame frame)

Optional. Allows subclass to push frame downstream in whatever shape or form it deems appropriate. If not provided, provided encoded frame data is simply pushed downstream.

proposeAllocation

protected boolean proposeAllocation(Query query)

Optional. Propose buffer allocation parameters for upstream elements. Subclasses should chain up to the parent implementation to invoke the default handler.

reset

protected boolean reset(boolean hard)

Optional. Allows subclass (encoder) to perform post-seek semantics reset. Deprecated.

setFormat

protected boolean setFormat(VideoCodecState state)

Optional. Notifies subclass of incoming data format. GstVideoCodecState fields have already been set according to provided caps.

sinkEvent

protected boolean sinkEvent(Event event)

Optional. Event handler on the sink pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler.

sinkQuery

protected boolean sinkQuery(Query query)

Optional. Query handler on the sink pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.4

srcEvent

protected boolean srcEvent(Event event)

Optional. Event handler on the source pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler.

srcQuery

protected boolean srcQuery(Query query)

Optional. Query handler on the source pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.4

start

protected boolean start()

Optional. Called when the element starts processing. Allows opening external resources.

stop

protected boolean stop()

Optional. Called when the element stops processing. Allows closing external resources.

transformMeta

protected boolean transformMeta(VideoCodecFrame frame,
 Meta meta)

Optional. Transform the metadata on the input buffer to the output buffer. By default this method is copies all meta without tags and meta with only the "video" tag. subclasses can implement this method and return true if the metadata is to be copied. Since: 1.6