Droidcon NYC 2014: Building Custom Camera Applications

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


Presentation slides from Droidcon NYC 2014: Building Custom Camera Applications

Transcript of Droidcon NYC 2014: Building Custom Camera Applications

  • 1. BUILDING CUSTOMCAMERA APPLICATIONSDroidcon NYC 2014Presented by Huyen Tue DaoSeptember 21, [email protected]+id/randomly_typing

2. BUILDING A CUSTOM CAMERA APPLICATIONWhere to startCamera setupImage captureCamera informationCamera parametersCamera API in L2 3. ABOUT MEMobile developer: native Androidand native iOSAndroid developer 4 years(personally use one)Computer Engineering, Universityof MD, College ParkGamer (video, board, card,anything): mostly Dota 2, QuizUp3 2011 Adam C Beamish 4. WHERE TO STARTWhat you want vs what you needBalance having criticalfeatures with supportingtarget user baseWhat featuresthe API supportsdeciding minimum SDKusing Build.VERSIONWhat featuresthe device camera hasusing use-feature and PackageManagerquerying support level from the Camerabeing aware of manufacturer/device quirksWANTCANYou cant always get what you want.But if you try sometimes well you just mind findYou get what you need. 5. CAMERA SETUP: THE MANIFESTWithout any specifications, to install a device would require:a back-facing cameraa front-facing cameraauto-focusflashFrom docs: This will automatically enforce the manifest element forall camera features.If any of the above are not requirements, specify :5 6. CAMERA SETUP: CAMERA + PREVIEWThe order of thingsAdd a SurfaceView to your layoutfor the camera previewImplement a SurfaceHolder.Callback:surfaceCreated(), surfaceChanged(),surfaceDestroyed()Pass the callback to the SurfaceHolderinstance of the SurfaceViewGet android.hardware.Camerainstance via Camera.open()Call Camera.getParameters()and perform any initial setupwait for surface to be created: #surfaceCreatedCall Camera.setPreviewDisplay() with the SurfaceHolderCall Camera.startPreview() 7. CAMERA SETUP: CAMERA + PREVIEWOpen/release camera 8. CAMERA SETUP: CAMERA + PREVIEWImplementing SurfaceHolder.Callback 9. CAMERA SETUP: THINGS TO NOTEDid you speak the exact words?Must call Camera.setPreviewDisplay() after thesurface is createdOtherwise, no error, no previewPreview size changes must be made betweenCamera.stopPreview()/Camera.startPreview()all Camera.release() in onPause() to free up the cameraThe surface is destroyed when SurfaceView is View.INVISIBLE.9 10. IMAGE CAPTURECall Camera.takePicture(ShutterCallback, PictureCallback,PictureCallback, PictureCallback)The 3 PictureCallback parameters = 3 image formatsraw: uncompressedpostview: scaled, fully processedJPEG: compressedRaw and postview support depends on deviceShutterCallback (more or less) invoked the moment that camerasensor captures an imageCamera.takePicture() stops the camera preview; callCamera.startPreview() in/after picture taking callbacks10 11. IMAGE CAPTUREwith a JPEG callback 12. CAMERA INFO: SWITCHING IT UPCameraInfo: per device cameraorientation: angle of rotation when facing the camera for theimage to match the natural orientation of the devicefacing: front or backwhether the shutter sound can be disabledCamera.getCameraInfo(): camera IDs are 0-based indicesUse CameraInfo to swap between front and backUse PackageManager to check if a front camera exists if frontcamera is not required by Close the current camera before swapping12 13. CAMERA INFOFinding the right camera 14. 14 15. 15 16. Natural camera orientation (looking at the device from the front)Natural device orientation90Camera.Info.orientation = 90270Camera.Info.orientation = 270DEVICE ORIENTATION VS CAMERA ORIENTATIONi.e. your camera is sideways 17. 17 18. CAMERA ROTATIONCamera/display rotation + configuration changes -> sucksComplicatedDevice and camera orientation changes dont play niceOption #1: change activity orientation animation (API 18+):WindowManager.LayoutParams.html#rotationAnimationOption #2: lock the activity orientation18 19. CAMERA PARAMETERSCamera.Parameters information about and access to featuresDevice support:PackageManager to check for auto-focus and flashOtherwise check getter return values in Camera.Parameters!isAutoWhiteBalanceLockSupported()Or returns 0 or nullAPI level also a factor (of course)19 20. CAMERA.PARAMETERS SETTING 21. CAMERA SETTINGS: UPDATINGCamera.Parameters changes happen immediately (mostly)Saving/restoring parametersCamera.Parameters.flatten()Camera.Parameters.unflatten()Change values on a Camera.Parameters instance and pass it toCamera.setParameters()Always call Camera.getParameters()Do not hold onto Camera.Parameters instancesOnly flattened string makes it past Camera service21 22. 22 23. CAMERA FEATURES: AREASCamera.Area rectangular bounds within viewfindermetering areas: measuring light to calculate exposure + WBfocus areas: prioritizing focusCamera.Parameters.setMeteringAreas(List)Camera.Parameters.setFocusAreas(List)Relative to the current zoom: cannot be outside FOVCamera viewfinder/sensor has its own coordinate system23 24. Camera-1000-1000 1000(0,0)H1000WViewViewfinderFull FOVViewfinderVIEWFINDER VS VIEW COORDINATESTranslating between what the camera sees and what the user seeszoom = 1.5xpreview surface in layout 25. 25 26. FACE DETECTIONCamera.Parameters.getMaxNumDetectedFaces()CameraFaceDetectionListener, setFaceDetectionListener()startFaceDetection(), stopFaceDetection()Camera.Face: data objectID: unique per faceface boundsscore: how confident are we that this is a face?eye and face coordinates26 27. FACE DETECTIONFaceDetectionListener.onFaceDetection() is called frequentlyStart only when actually neededAvoid intensive work inside the callbackNo event when no faces are detectedModifying the UI in response to face detection: timer orpostDelayed to reset UI after some time without detection event27 28. CAMERA API 2 IN L: THE FUTURE IS AWESOMEDriven by a new Hardware Abstract Layer (HAL3)HAL Version 1: black box: 3 modes (preview, still, video)HAL3: all requests handled same way, unified viewMore user control for both capture and post-processingMaintainability and efficiency28 29. HAL V1 VS HAL V3https://source.android.com/devices/camera/camera3.html 30. HAL3 (EXPANDED)https://source.android.com/devices/camera/camera3_requests_hal.html 31. CAMERA API 2 IN L: THE FUTURE IS AWESOMEMore metadataCamera informationCapture configuration part of request and resultDynamic metadata: timestamps, exposure timeMore output optionsMultiple Surface instances receive results from a single requestApp-visible JPEG, YUV, RAW Bayer buffers31 32. CAMERA API 2 IN L: THE FUTURE IS AWESOMECamera (static bits) -> CameraManagerAvailabilityListener: removable camerasAsynchronous opening + CameraDevice.StateListenerCamera (function access) -> CameraDeviceActually get the ID of the camera!CameraCaptureSession/CaptureRequestParameters -> CaptureRequest.Builder.set() to configureCameraInfo -> CameraCharacteristicsCameraMetadata parent of CameraCharacteristics, CaptureRequest,CaptureResult, TotalCaptureResult32 33. CAMERA API 2 IN L: THE FUTURE IS AWESOMELessons from API 1 do not go away (remember the words)More complex but more is still betterMore control and performanceMore consistency in the APIMore for developers to leverageIts going to be33 34. THANK YOU +QUESTIONS?Huyen Tue DaoLead developer | OwnerRandomly [email protected][email protected]!speakerdeck.com/randomlytypingslideshare.net/randomlytyping34Code will coming to github!Things To check out!Building Apps with Multimedia: Capturing Photosdeveloper.android.com/training/camera/index.html!Android Design in Action: Camera Appsyoutube.com/watch?v=OLSa7fErTAM!Standford Digital Image Processing Classstanford.edu/class/ee368/Android/index.html!DevBytes: Android L - Camera2 APIyoutube.com/watch?v=Xtp3tH27OFs!Sample Camera 2 API app:github.com/googlesamples/android-Camera2Basic/blob/master/Camera2BasicSample!Digital Camera Sensor Sizesgizmag.com/camera-sensor-size-guide/26684/