Identify and analyze the visual aspects of the meeting along with the corresponding timestamps and with Screen Activity API.
Activity | Description |
Speaker/Interaction | when the meeting/conversation has no visual elements or if it is face-to-face (video is ON) conversation. |
Sketching/whiteboarding | detects when there is whiteboarding or sketching in the meeting |
Presentation | detects full-screen presentations during the meeting |
Screen Share | detects general screen share like browsing through the computer, webpages, etc. during a screen share session. |
Input Type Supported: Video, Image
Model Dependency: OCR
screengrabs.enable key is set to true under the settings objectTransaction ID is returned in the JSON body once the processing job is launched successfully.
This Transaction ID can be used to check the status of the job or fetch the results of the job once the metadata is computed{"status":true,"transaction_id":32dcef1a-5724-4df8-a4a5-fb43c047716b,"message": " Compute job for file-id: 32dcef1a-5724-4df8-a4a5-fb43c047716b launched successfully"}
Speech to Text has to be enabled for Action Items to be enabled).
This can also happen when the input file_id is not of the supported format as shown in the example below.{"status":false,"error":{"code":"VDNF01","message":"FileTypeError: Require file to be of type Video"}}
curl --request POST 'https://api.marsview.ai/v1/conversation/compute' \--header 'appSecret: 32dcef1a-5724-4df8-a4a5-fb43c047716b' \--header 'appId: 1ZrKT0tTv7rVWX-qNAKLc' \--header 'Content-Type: application/json' \--data-raw '{"settings":{"speech_to_text":{"enable":true,"pii_detection":false,"custom_vocabulary":["Marsview" , "Pikachu"]},"speaker_separation":{"enable":true,"num_speakers":4},"screen_activity":{"enable":true}}}'
Given below is a sample response JSON when the Status code is 200.
{"status":true,"transaction_id":32dcef1a-5724-4df8-a4a5-fb43c047716b,"message": " Compute job for file-id: 32dcef1a-5724-4df8-a4a5-fb43c047716b launched successfully"}
data object returns the requested metadata if it is computed. The status object shows the current state of the requested metadata. Status for each metadata field can take values "Queued"/"Processing"/"Completed".
Shown below is a case where "sentiment analysis" Job is in "Queued" state and "Completed" state. {"status":{"screen_activity":"Queued",}"data":{"screen_activity":{}}}
{"status":{"screengrabs":"Completed"}"data":{"screengrabs":{"chunks":[...{"frame_time" : "174100.0""frame_id" : 1235"confidence" : "0.33","type":"Presentation"},{"frame_time" : "174100.0""frame_id" : "1521","confidence" :"0.95","type":"Screen Share"},...]}}}
Fields | Description |
| Offset time of the frame in milliseconds from the starting of the video |
| Offset frame_id (Frame number) from the starting of the video |
| Type of the screen that is detected. refer to Types for more details. |
| Confidence of the speech type label (ranges from 0 to 1). Higher the better |