Screen Activity

Overview
Identify and analyze the visual aspects of the meeting along with the corresponding timestamps and with Screen Activity API.
Types
Activity
Description
Speaker/Interaction
when the meeting/conversation has no visual elements or if it is face-to-face (video is ON) conversation.
Sketching/whiteboarding
detects when there is whiteboarding or sketching in the meeting
Presentation
detects full-screen presentations during the meeting
Screen Share
detects general screen share like browsing through the computer, webpages, etc. during a screen share session.
Speaker Face
Detect when the speakers face is displayed in the meeting.
modelTypeConfiguration 
Keys
Value
modelType
screen_activity
modelConfig
Model Configuration object for screen_activity
​
Example Request
Curl
Python
curl --location --request POST 'https://api.marsview.ai/cb/v1/conversation/compute' \
--header 'Content-Type: application/json' \
--header "authorization: {{Insert Auth Token}}" \
--data-raw '{
        "txnId": "{{Insert txn ID}}",
        "enableModels":[
            "modelType":"screengrabs",
            "modelConfig":{
                "ocr":{
                    "enable":true
                    },
                "screen_activity" : {
                    	"enable" : true,
                    	"classes" : ["drawing","ppt","screen',"speaker_face","speaker_presentation"]
                        }
                }
            },
            {
            "modelType":"screen_activity"
            }
        ]
}'
import requests
auth_token = "replace this with your auth token"
txn_id = "Replace this with your txn id"
request_url = "https://api.marsview.ai/cb/v1/conversation/compute"
​
​
def get_screen_activity():
  payload={
        "txnId": txn_id,
        "enableModels":[
            {
            "modelType":"screengrabs",
            "modelConfig":{
                "ocr":{
                    "enable":True
                    }
                }
            },
            {
            "modelType":"screen_activity"
            }
            ]
        }
  headers = {'authorization': '{}'.format(auth_token)}
  
  response = requests.request("POST",headers=headers, json=payload)
  print(response.text)
  if response.status_code == 200 and response.json()["status"] == "true":
    return response.json()["data"]["enableModels"]["state"]["status"]
  else:
    raise Exception("Custom exception")
​
if __name__ == "__main__": 
    get_screen_activity()
Example Response
"data": {
    "meetingActivity": [
    {
        "shotId": 1,
        "startFrame": 1,
        "endFrame": 3237,
        "meetingActivity": [
            {
                "className": "speaker_fused"
            }
        ],
        "meetingActivityAccuracy": {
            "speakerFace": "3.0225227"
        },
        "startTime": 66.66666666666667,
        "endTime": 215800
    }
}
Response Object
Field
Description
meetingActivity
List of meeting activity objects
shotId
Id of the collection of frames being analyzed
startFrame
First frame of the specified shotId
endFrame
Last frame of the specified shotId
meetingActivity[meetingActivity]
Type of meeting activity identified in the shotId
className
Type of class best describing the type of screen activity
meetingActivityAccuracy
Object speicying the accuracy of the predicted meeting activity
speakerFace
​
startTime
Start time of the shotId in milliseconds
endTime
Start time of the  shotId in milliseconds

Summary

Screengrabs

Last modified 11mo ago

Copy link

Outline

Overview

Types

modelTypeConfiguration

Example Request

Example Response

Response Object