Bebop 2 v4.7.1 camera / ISP / vision chain (static analysis only)
Paths below are relative to .../firmware/extracted/bebop-2-4.7.1/rootfs (RFS). Ghidra address = ELF file offset + 0x10000 for .rodata strings in usr/bin/dragon-prog (e.g. file offset 0x48ac bc -> Ghidra 0x49acbc). Nothing was executed; Ghidra was used read-only.
Label key: EVIDENCE = directly observed (file + address/string). INFERENCE = my reading.
0. Headline corrections to the working assumptions
Section titled “0. Headline corrections to the working assumptions”- EVIDENCE:
/calibis NOT empty in this rootfs. It holds 13 files (section 5): lens F-theta tables, ISP tuning curves, a camera EEPROM image. Only/dataand/factoryare empty. - EVIDENCE: DxO2a (“Impala” ISP library), Alpaga (3A), Vision360 (fisheye/GPU reprojection), libVision2 and libcam-angles are NOT separate .so files. They are statically linked into
usr/bin/dragon-prog(source paths in strings,etc/manifest.xmllists libDxO2a, libVision2, libVision360 as git projects). - EVIDENCE: libCondor is the optical-flow / visual translation engine for the vertical camera, not the video stabilizer. Exports
condor::Engine::{setBuffer,Compute,getTranslation,getIndicator,setAttitude,setCamera,setCameraGain,setParamsDimensions,setParamsQuality,setParamsPrediction,getCorrectionCameraOffset,Init}(dragon-prog string table, imports at EXTERNAL:0x63..0x344;nm -D usr/lib/libCondor.so). Video stabilization is libcam-angles + libVision360 + the videoWindowSetter thread (section 3). - EVIDENCE: optical flow is software (CPU, Eigen based) but runs on a 160x120 luma image; there is no flow hardware block referenced.
1. Inventory
Section titled “1. Inventory”1.1 Sensors and hardware (from scripts and strings)
Section titled “1.1 Sensors and hardware (from scripts and strings)”| Item | Evidence | Role |
|---|---|---|
| MT9F002 (onsemi/Aptina 14 MP, 4608x3288 active, 1/2.3”) | usr/bin/mk3_cam0_common.sh (MT9F002_I2C_DEVICE="0x10", chip_version 0x2e01 comment), HAL_MT9F002_* symbols in dragon-prog, P3A XML width 4608 height 3288 bitDepth 10 |
Main (“horizontal”, H) front camera, I2C bus /dev/i2c-0, MCLK 13 MHz PWM (etc/init.d/rcS_video, period 77 ns) |
| MT9V117 (VGA SoC, 0x5D) | usr/bin/mk3_cam2_common.sh (MT9V117_I2C_DEVICE="0x5D", chip id 0x2282 comment), usr/bin/diagnostic strings init_mt9v117, mt9v117 patch apply failed |
Downward (“vertical”, V) camera, MCLK 43 MHz (rcS_vision, period 23 ns) |
| Camera EEPROM | usr/bin/mk3_camera_eeprom, /calib/eeprom_cam.dat header MK3CAMERADATA |
Per-unit camera data, 8 KiB image |
| Gyro FSYNC GPIO | rcS_video: exports FSYNC_GYRO_P7, edge=rising, “configure phase module to start counting at frame start (rising Vsync)” |
Ties sensor frame start to IMU timeline (stabilization sync) |
| Parrot P7 SoC video blocks | kernel modules lib/modules/3.4.11+/kernel/drivers/parrot/media/video/{avicam,avi_v4l2,avi_r2r,avi_m2m,avi_stats,avicam_dummy_dev}.ko, parrot/video/{avicore,avifb}.ko, parrot/gpu/{mali400/mali.ko,ump/ump.ko,p7ump.ko,p7gpu-pm.ko} |
AVI = camera interface + ISP + scalers (V4L2 devices), Mali-400 GPU |
EVIDENCE for sensor part numbers: MT9F002, MT9V117 (above). No lens/sensor part string for the exact module, but lens table names DSL219/DSL239/DSL217 (Sunex fisheye lens model numbers, inference on vendor) and strings meta_calib_bebop_sunex, meta_calib_bebop_suny (dragon-prog 0x1719c, 0x17749) exist.
1.2 Libraries (RFS/usr/lib, with readelf -d NEEDED)
Section titled “1.2 Libraries (RFS/usr/lib, with readelf -d NEEDED)”| Library | Size | NEEDED (non-libc) | What it does |
|---|---|---|---|
| libh1enc.so | 90 KB | none | Hantro H1 hardware H.264 encoder API (EVIDENCE: H264CodeFrame, H264Init, EncAsicFrameStart, EWLInit, EWLWaitHwRdy, H264InitRc, H264SeqParameterSetInit, H264EncSetRateCtrl/CodingCtrl/PreProcessing/SeiUserData imported by dragon-prog). Rate control and bitstream (SPS/PPS/slice) generation in software around an ASIC. |
| libvideo-encode.so | 39 KB | libvideo-buffers, libvideo-streaming, libh264, libvideo-buffers-p7, libh1enc | venc_* wrapper, venc_hantro_h1_* backend, SEI writers (venc_h264_sei_add_recovery_point, ..._picture_timing, ..._parrot_streaming_v2_user_data). |
| libvideo-buffers.so / -p7.so | 36 KB / 6 KB | libpomp, libh1enc | vbuf refcounted buffers; p7 variant backs them with EWL linear (physically contiguous) memory (vbuf_p7_alloc -> EWLMallocLinear). |
| libh264.so | 275 KB | json-c | H.264 NAL/SPS/PPS/SEI parse and write (h264_parse_sps, h264_write_nalu). Also contains a DxO string. |
| libframeinfo.so | 36 KB | libm | Parrot FrameInfo V1/V2/V3 and Video Streaming/Recording Metadata V1/V2 (de)serialisers, plus H.264 user-data SEI wrappers (FRAMEINFO_SerializeH264UserDataSeiParrotFrameInfoV3). |
| libvideo-metadata.so | 68 KB | futils, json-c | vmeta session date/location and per-frame metadata (carries the com.parrot.videometadata2 MIME seen in dragon-prog 0x49da4). |
| libvideo-streaming.so | 79 KB | metadata, librtp, libh264 | Stream/AU handling for the live path. |
| librtp.so | 23 KB | libpomp | RTP packetization primitives. |
| libarstream2.so | 333 KB | libarsal, libarmedia | ARSDK3 stream v2 sender/receiver (ARSTREAM2_StreamSender_Init, _SetDynamicConfig, _SetUntimedMetadata, ARSTREAM2_H264Parser_*, ARSTREAM2_H264Writer_*, ARSTREAM2_H264Sei_SerializeUserDataParrotStreamingV1/V2). RTP over UDP with control channel. |
| libarstream.so | 22 KB | arnetwork | legacy ARStream v1 (present but dragon-prog needs v2 only). |
| libsuperstream.so, libdragonvideo.so | 42 / 49 KB | arstream2, video-encode, video-buffers, video-metadata, telemetry | Glue used by video telemetry/streaming (NEEDED chain shows libdragonvideo -> arstream2 + video-encode). INFERENCE on exact role: dragon-prog itself does not NEED libdragonvideo, so this is likely used by the ARSDK dev/test tools or a legacy path. |
| libpicture-file.so | 178 KB | armedia, pal-core, exif, turbojpeg, jpeg, tiff | JPEG/DNG writer + EXIF (dragon-prog strings DNG: writing..., PHOTO_MODE_*). |
| liboyster.so | 7 KB | none | Small filter library: oyster_set_position_limits / velocity_limits / smoothness / filter_coefs / filter_process. Used by videoWindowSetter for the user camera orientation (pan/tilt) smoothing (dragon-prog: “failed to initialize oyster context for user orientation” 0x48fb4c, videoWindowSetter_getCameraUserOrientation). It is the digital-gimbal command smoother. |
| libCondor.so | 114 KB | stdc++ only | Optical-flow (translation) engine plus DumpWriter/DumpReader for recording vcam frames and attitude (CONDOR * STARTING VCAM RECORDING). Uses Eigen. Also has condor::Prediction::{update,getPrediction,shift_image}: motion prediction used to shift the search image. |
| libMali.so, libUMP.so, libEGL/GLESv1/v2 | Mali-400 GL ES 2.0 userland. Used by Vision360 GPU reprojection (shaderProgramLuma/Chroma/LumaTNR/ChromaTNR/HalfTNR, pixmapEGL.c, p7umplib.c). |
||
| libion_parrot.so, libmedia-ctl.so, libmeschach.so | ION allocator, V4L2 media-controller helper, matrix library (meschach, used by vision/estimation). | ||
| pal-drivers.so, pal-core.so, pal-utils.so | Parrot abstraction layer; etc/PAL/priority.map sets thread priorities (see below). |
Statically linked into usr/bin/dragon-prog (6.1 MB, EVIDENCE: source file strings):
packages/DragonProg/Sources/features/video/ardrone3/{video_dxo.c,video_platform_utils.c,photo_platform.c,photo_record_platform.c,thread_video/fisheyeCalibLoader.c}packages/libcam-angles/Sources/{camAngles.c,gyroIntegration.c,gyroResample.c,resample.c(+neon),firFilt.c,expFilter.c,windowFiltfilt.c,quat.c,ringBuffer.c}packages/libVision360/FisheyeLib/Sources/{CamWindows/camWindows.c, Projection/FisheyeCam/RTheta.c, Projection/RectilinearCam/distortion.c, Projection/Mesh/mesh.c, Projection/Wobble/{gyrosFilter,gyrosIntegration,resample}.c, YUV420SP_core/*, YUV420SP_GPU_core/*, GPU/{glesTool,gpuInit,pixmapEGL,shaderTool,UMP/p7umplib}.c, Blending/blendMedian.c, fisheye_XYLut.c, fisheye_angleSamples.c}packages/libHAL/Sources/{HAL_v4l2_utils.c,HAL_i2ctool.c,...},HAL_MT9F002_*driverpackages/Viz/src/image/filter.cpp- DxO2a/Impala ISP and Alpaga 3A: strings
Impala:%hhu.%hhu P3A:%hhu.%hhu(0x48c41c),libImpala:15ea8dec14f98c394fca9ce6a707382b43221db9,ts_Impala_Gain_Tuning,==== P3A version,__ulog_cookie_Alpaga_{AE,AWB,statools}.
1.3 Configs and scripts
Section titled “1.3 Configs and scripts”| File | Role |
|---|---|
etc/P3A_ctx_init.xml |
Alpaga AE/AWB context (section 2) |
etc/colibry/common/video.cfg + mykonos3board/video.cfg |
digital gimbal tilt limits and stabilization defaults (section 2) |
etc/default-dragon.conf |
default user video/photo settings |
etc/init.d/rcS_video, rcS_vision |
camera power-down GPIO, MCLK PWM, gyro FSYNC edge |
usr/bin/mk3_cam_common.sh, mk3_cam0_common.sh, mk3_cam2_common.sh |
i2ctool helpers to read/write MT9F002/MT9V117 (shell debugging) |
usr/bin/mk3_camera_eeprom, check_hcam_pinout, check_sensors, diagnostic, p7_sensors-test |
factory tools (camera EEPROM read/write, pinout check, sensor ID checks). Not run. |
etc/PAL/priority.map |
v4l:/dev/video* -19, videoWindowSetter -18, Vision/Vision_start -10, video_live -4, VideoMain -4, video_reproj -4, camAngles -4, video_rec -1 (lower = higher priority). Shows the thread set. |
etc/colibry/common/root.cfg |
SensorCarpetProcessing = "info", SensorOpticalFlowProcessing = "minimal" (flight-controller side logging verbosity for those sensor processors; EVIDENCE lines 25, 32) |
2. Parameters
Section titled “2. Parameters”2.1 Alpaga AE/AWB (etc/P3A_ctx_init.xml, read in full)
Section titled “2.1 Alpaga AE/AWB (etc/P3A_ctx_init.xml, read in full)”| Parameter | Value | Note |
|---|---|---|
| bitDepth | 10 | sensor RAW bits |
| width x height | 4608 x 3288 | full MT9F002 array |
| nbAECallPerSec | 30 | AE runs at frame rate |
| AEFrameDelay | 1 | sensor exposure applies one frame late |
| nbAWBCallPerSec | 7 | |
| GammaTable | 1024 entries, 0..255 | 10-bit in, 8-bit out; steep toe (0,1,3,5,6,8,…), reaches 252-255 only at the top |
| LumaTarget | 141 | 8-bit Y target after gamma (about 55%) |
| AE_LUT | 256 weights (0 first and last, 1 elsewhere) | histogram weighting; ignores bin 0 and 255 |
| TG_curve exposure (ms) | 0.2, 3, 3, 4, 4, 10, 10 | piecewise-linear path through exposure/gain space |
| TG_curve gain | 1.5, 1.5, 4, 4, 8, 8, 16 | so: exposure 0.2->3 ms at gain 1.5, then gain 1.5->4 at 3->4 ms… up to 10 ms and gain 16 |
| Preview_smoothing | dampen 0.1, speed 8, smoothness 0.9 | |
| Clipping | darkClip 16, highlightClip 239, dark ratio 0.10, highlight ratio 0.25 | |
| AWB fooConvergence | 0.05 | |
| AWB luma | usLowMaxRGB 30, usHigMaxRGB 850, weights 10/10 | |
| AWB presets (R scale, B scale) | 0 Horizon 228/649; 1 Tungsten 260/574; 2 TL84 300/480; 3 Fluo/CWF 350/500; 4 Sunny 0/0 (unset); 5 D55 345/400; 6 D65 360/330; 7 Shade 0/0 (unset). Sub-colours for 0, 2, 5, 6 add skin/sky/etc style anchor points (e.g. D55 type0 385/280, type2 435/606; D65 type0 420/220, type2 465/500) | usGreyR2 2000-4500 |
| debug paths | /data/ftp/internal_000/Debug/Alpaga_dbg.txt, alpaga_lscMap.bin, alpaga_Histo.bin, etc. |
emode 0 |
EVIDENCE note: the XML says “The gamma table … should be read from calib data” (comment), and dragon-prog has GammaTable, usXcellsizeLSC strings and the message XXX KEEPING DEFAULT LSC GAINS / LSC GAINS USED: %f %f %f after trying /data/awb_gains.txt (0x4923d0). So per-unit lens-shading gains live in /data/awb_gains.txt.
2.2 Video modes (table in dragon-prog .rodata, file offsets 0x490a20..0x4911f0, Ghidra +0x10000)
Section titled “2.2 Video modes (table in dragon-prog .rodata, file offsets 0x490a20..0x4911f0, Ghidra +0x10000)”Each record starts with an fps float. EVIDENCE: values decoded from the binary.
| fps (float) | Two floats | Encode buffer / crop | Stream size | Record size | sensor-window rows x cols |
|---|---|---|---|---|---|
| 29.97 | 78.0, 49.0 | 1920x1088 buffer, 1920x1080 | 640x368 (alt table: 864x480 with 856x480 crop) | 1080p | 1600 x 2112 |
| 25 | 80.0, 50.5 | same | same | 1080p | 1600 x 2112 |
| 23.976 | 80.0, 50.5 | same | same | 1080p | 1344 x 2112 |
| 29.97 / 25 / 23.976 (720p family) | same pairs | 1280x720 | 1280x720 | 720p | 1600/1600/(garbled) x 2112 |
Each record also has four sub-entries (1600 or 1344, 2112, then floats 1.0/1.0/0/0.92/0.97, 0.89/1.0/0/0.90/0.95, 0.762/1.0/0/0.90/0.95, 0.5/1.0/0/0.90/0.95). INFERENCE: these are the four zoom/stabilization window scale levels of the digital gimbal (1.0, 0.89, 0.762, 0.5 of the nominal sensor window), but I did not find the code that consumes the final two floats. INFERENCE: 2112x1600 is the sensor read window after 2x binning (4608/2 = 2304, 3288/2 = 1644, so 2112x1600 is a crop inside 2304x1644, leaving about 192x44 px of margin for stabilization shift). The 1920x1080 crop then comes from that 2112x1600 window.
EVIDENCE for the strings that consume this: buildAcquisitionWindow, videoWindowSetter ... window dimensions (%d;%d) exceed sensor dimensions (%d,%d), HAL_MT9F002_SetPosition, failed to set res/fps for sensor conf #%d, unknown binning mode %d for sensor conf #%d, failed to get timing for sensor conf #%d (0x4a0230..). The sensor conf builder is FUN_000960a0 (reads w,h, divides by scalers at struct offsets +0xc/+0x10, doubles on binning).
User settings: etc/default-dragon.conf has video_stabilization_mode 0, video_recording_mode 1, video_frame_rate 2, video_resolution_mode 0, picture_format 2, video_autorecord true, max_camera_velocity_tilt/pan 20.0 (deg/s, inference on unit). dragon-prog usage string: -f fps forced video framerate; can be 30 (forced to 29.97), 25 or 24 (forced to 23.976); -b record bitrate Kbit/s; -q max stream bitrate; -s disable bitrate adaptation; -H stream mode (0 low latency, 1 high reliability, 2 high reliability low fps); -S stabilization on/off; -R reprojection mode off/cpu/gpu; -m/-x/-y/-z/-E/-A/-P DxO AE/AWB overrides; -D dump VCam frames.
Photo modes (strings 0x4922f8..): FULLFRAME_RAW_BAYER_10b/12b, DXO, DNG10, DNG12 (“isn’t supported yet”), JPEG43, SNAPSHOT, THUMBNAIL, FISHEYE_JPEG.
2.3 Digital gimbal / stabilization config
Section titled “2.3 Digital gimbal / stabilization config”etc/colibry/common/video.cfg: videoAnglesFlat=false, videoStabCoef=1.0 ([0,1]), videoStabilizationMode=0 ({0 Flat, 1 Pitch, 2 Roll, 3 None}), videoTiltRange=[-75,30] deg. mykonos3board/video.cfg overrides videoTiltRange=[-70,35] (this is the Bebop 2 board; milosboard/chimeraboard have their own video.cfg). The ARSDK commands ARDrone3PictureSettingsVideoStabilizationMode, ARDrone3CameraOrientation reach this.
2.4 Lens and calibration (all in /calib, section 5)
Section titled “2.4 Lens and calibration (all in /calib, section 5)”F-theta tables are two-column field angle (deg) image height (mm):
DSL219-FThetaLUT.dat91 rows, 0..90 deg, up to 2.626 mmDSL239-FThetaLUT.dat88 rows, 0..94.08 deg, up to 2.678 mm (dragon-prog hardcodes this path, 0x492a44)SunnyMilos-FthetaLUT.dat106 rows, 0..98 deg, up to 2.696 mm (hardcoded 0x492a94)FThetaLUT_DSL217.dat927 rows, 0..92.5 deg, up to 1.91 mmcustom_43_disto_lut.dat100 rows, 0..60 deg, up to 1.921 (hardcoded 0x48c2bc, with “Failed to load calib %s. Use a default.”; used byPHOTO_MODE_JPEG43rectilinear 4:3 projection,distortion.c)
INFERENCE: per-lens selection through fisheyeCalibLoader_loadLensType / loadRtheta / loadEllipse; the Bebop 2 lens maps to DSL239 (hardcoded path in the Bebop-targeting code path) while SunnyMilos is the Milos (Disco/Bebop-family board) lens. Not proven which one a given unit uses; it is selected by /data/eeprom.dat content (the loader prints LOAD EEPROM DUMP FILE : %s 0x492e9c).
3. The chain
Section titled “3. The chain”3.1 Main (front) camera, live + record
Section titled “3.1 Main (front) camera, live + record”- MT9F002 sensor (EVIDENCE: HAL_MT9F002, I2C 0x10). Configured by
HAL_MT9F002_Init, PLL params, binning andHAL_MT9F002_SetPosition(window position written per frame byvideoWindowSetter). MCLK 13 MHz (rcS_video). Powered viaCAMERA_H_PWDNGPIO. FSYNC to gyro. - P7 AVI camera interface to V4L2 (
avicam.ko,avi_v4l2.ko) into physically contiguous buffers. dragon-prog opens V4L2 devices throughHAL_v4l2_utils(“failed to start v4l2 client”, “enable delegate mode for ‘%s’”). Device roles by string adjacency (INFERENCE):/dev/video1main capture (VIDEO_ThreadHandler region: “failed to load calibration datas”,/data/eeprom.dat),/dev/video2Alpaga statistics (thread_dxo, video_dxo.c),/dev/video3streaming scaler (video_stream->... scaler input/output),/dev/video4RAM2RAM ISP (photo pass),/dev/video5RAM2RAM scaler (photo thumbnail),/dev/video0= vision (V) camera,/dev/media0for media-controller graph setup. EVIDENCE: the strings and their order; the numeric mapping is inference. - ISP: the P7 AVI hardware pipeline configured by the Impala library (DxO2a): Bayer to YUV with DPC, noise reduction, color correction, edge enhancement and gamma. The tuning tables are
/calib/ftd_dpc.txt(defective pixel list),ftd_nr.txt,ftd_cc.txt,ftd_curves.txt,ftd_ee.txt,userTuning*.txt(section 5). Log line:[initPerUnitSettings_dxo] Load per unit data.and[initPerUnitSettings_dpc]. EVIDENCE for the stage names: only the file suffixes and thevideo_ee_*thread (EE = edge enhancement is INFERENCE; the thread namevideo_eehas input/output buffers and an “ee ram2ram” ISP init). - Alpaga 3A: statistics from the AVI stats block (
avi_stats.ko, histogram,maxRGB,weightMap,corrStatsdebug dumps) feed AE (exposure ms + analog gain + digital gain throughHAL_MT9F002_*,Video_DxO2A_ForceExp/ForceGain/WBGains/setEvBias/setAEMode/setElectricFrequency) and AWB (R/B gains).thread_dxoprints**** DxO2a Initialized/running. Anti-flicker viasetElectricFrequency(50/60 Hz). - Stabilization window:
camAnglesthread (libcam-angles: gyro integration, resample, FIR/filtfilt, quaternion) fuses gyro with the drone attitude to produce per-frame camera orientation.video_imu_frame_syncaligns gyro samples to the frame’s SOF/vsync (“failed to find SOF in gyro data”, “failed to synchronize vsync and camAngles data”).videoWindowSetterpredicts a crop window (“predictAngles/predictReqWindow/predictSetWindow/stabAngles/stabReqWindow/stabGetWindow” debug lines, 0x4900c4..) and moves the sensor readout window; liboyster smooths the user’s tilt/pan command. Late application is detected (“setting applied too late”). - Fisheye reprojection / de-wobble (
video_reprojthread): per-frame crop pos and exposure/gain logged (“crop pos (%d,%d)”, “exp %2.2fms gain: %1.2f”). Residual rotation is removed by libVision360 using the F-theta LUT, mesh warping and gyro-derived wobble filter, either on CPU or on the Mali GPU (-R off|cpu|gpu;pixmapEGL,shaderProgramLuma/Chroma,...TNR= temporal noise reduction variants). Output is NV12/YUV420SP in EWL linear buffers. - Splitting: one frame goes to (a) the full-size recording encoder, (b) a scaler to the stream size (640x368 or 856/864x480 or 1280x720). EVIDENCE strings: “failed to lock both scaler and encoder buffers”, “failed to get scaler input buffer. A live frame will be dropped”, “video_stream_encodeFrame()”.
- H.264:
P7_H264_*wrappers over libh1enc (H264EncInit,H264EncSetRateCtrl/CodingCtrl/PreProcessing,H264EncStrmStart,H264EncStrmEncode) at Hantro H1 hardware (EWL). Live stream bitrate adapts to Wi-Fi (videoStreamingTargetBitrate,ARSTREAM2_StreamSender_SetDynamicConfig, “video streaming config dynamically changed”). - Per-frame metadata: FrameInfo V3 and VideoStreamingMetadata via H.264 user-data SEI (
FRAMEINFO_SerializeH264UserDataSeiParrotFrameInfoV3,ARSTREAM2_H264Sei_SerializeUserDataParrotStreamingV1/V2), plusstabmetadataandvideo_telemetry(the.264NAL parser/writer rewrites non-reference P slices:ARSTREAM2_H264Writer_RewriteNonRefPSliceNalu,video_stream_fixNonRefFrame). - Network: libarstream2 StreamSender (RTP over UDP; ports negotiated in the ARSDK connection JSON:
arstream2_server_stream_port,arstream2_server_control_port,c2d_*ports, dragon-prog 0x4894b8). Also-Fcan dump the H.264 to/data/ftp/internal_000/stream/stream_%03d.264. - Recording branch:
video_recthread,ARMEDIA_VideoEncapsuler(MP4 withapplication/octet-stream;type=com.parrot.videometadata2metadata track, SPS/PPS viaP7_H264_getSpsPps, thumbnail, autocut by size, “time between two encoded frames > 50ms” watchdog). - Photo branch: pauses the video thread (“THREAD VIDEO: Paused by photo”), reconfigures the sensor for full or fisheye frames, runs the RAM2RAM ISP (
/dev/video4), optional reprojection, then JPEG or DNG via libpicture-file.
3.2 Downward (vertical) camera and vision navigation
Section titled “3.2 Downward (vertical) camera and vision navigation”- MT9V117 on I2C 0x5D, MCLK 43 MHz,
CAMERA_V_PWDN, V4L2/dev/video0(EVIDENCE: dragon-prog 0x4890b0 in the Vision start code andDefective vertical camera detected,Vertical Camera KO). - Vision thread (
VISION_ThreadHandlerGhidra 0x79568): reads propertypersist.dragon-prog.vision_fps(0 = max), dequeues frames from V4L2, optionally rate-limits to 1000/fps ms, and for each accepted frame calls the Condor wrapperFUN_0003fdb8(EVIDENCE). Every 4th frame it also reads a value (FUN_0044e950) intoFUN_000404c8(INFERENCE: ultrasound/altitude or baro scale for translation scaling). Failure after 4 bad frames marks the sensor defective (BEHAVIOUR_FEATURE_VISION_MSG_DEFECTIVE_SENSOR_DETECTED). - Vision context (
FUN_00079e5c): pixel format 0x59565955 = UYVY, 320x240 (0x140 x 0xf0), V4L2 buffer type fields (EVIDENCE: decompile). NoteFUN_0003fdb8also accepts 0x56595559 = YUYV. - Condor init (
FUN_0003fa9c):setParamsDimensions(320,240,160,120,5,5,3,2), so the 320x240 frame is reduced (viz::Reduce_Picture_ItoP) to a 160x120 8-bit luma buffer (0x4b00 = 19200 bytes, two of them, ping-pong swap at function end).setParamsPrediction(...,1.0),setParamsQuality, andsetCamera(fx=153.5, fy=153.5, skew 0, cx=80, cy=60, 0, 0, 100.0, R, T). EVIDENCE for the values: little-endian floats read from the literal pool at Ghidra 0x3fd90..0x3fdac (0x43198000=153.5, 0x42a00000=80, 0x42700000=60, 0x42c80000=100). The parameter-to-field mapping is INFERENCE from the decompiled argument order. The rotation matrix read at Ghidra 0x498f78 is[[0,-1,0],[1,0,0],[0,0,1]](EVIDENCE of data), INFERENCE: camera-to-body 90 degree rotation about the optical axis. The 6 distortion coefficients are all zero (EVIDENCE: zero-filled temp). - Per frame:
Engine::setBuffer(new 160x120 luma),Engine::setAttitude(roll,pitch,yaw + delta terms)from the estimator (gyro-derotation),Engine::Compute, thengetTranslation(x,y,z, inferred pixel translation; z from scale),getIndicator(quality) andgetCorrectionCameraOffset. Results are scaled by a value read from the estimator (/ fVar20, INFERENCE: altitude) and pushed withFUN_000d0374into the shared vision data structure that the estimator reads. Names exposed to the rest of the system (dragon-prog 0x4a2248..):flow_x,flow_y,flow_z,vision_defined,vision_indicator; Libvar typesDataFromVision,VisionCarpetDetection(sensors.visionCarpetData.nb_detect,.orientation);VISIONRES_condor_{tx,ty,tz,defined}. - Debug:
/data/ftp/internal_000/condor.txttiming log (via property flag),-D framesWait:framesDumpdumpsvid+att_*.uyvy422,CONDOR * STARTING VCAM RECORDINGwrites dumps withDumpWriter. - Carpet:
VisionCarpetDetection/Mykonos3::ControlStateDemoMagicCarpet,demoMagicCarpetinetc/colibry/common/states.cfg. INFERENCE: a demo mode that detects a known patterned carpet; the detector itself is not in libCondor’s export list, so I did not locate the algorithm. - Fusion: handled by the other agents’ scope (Kalman filter
estimator.cfg). EVIDENCE of interface only: the names above plusACQCON_DATA_EXPORT.
4. Per-unit calibration expected at runtime
Section titled “4. Per-unit calibration expected at runtime”| Path | EVIDENCE of use | What it would hold |
|---|---|---|
/data/eeprom.dat |
dragon-prog 0x48ba28, “LOAD EEPROM DUMP FILE to read camera serial number” | Copy of the camera module EEPROM (8 KiB, MK3CAMERADATA format) that the app uses for camera serial and lens/projection data. mk3_camera_eeprom can read/write the real EEPROM over I2C. |
/data/awb_gains.txt |
0x4923d0 | Per-unit lens-shading/white balance gains (“LSC GAINS USED: %f %f %f”); default when missing |
/factory/calib.json |
0x49fe10 | Factory calibration (likely IMU/magnetometer/accessory, not proven camera) |
/factory/serial.txt |
0x47aa8c | drone serial |
/factory/country_setting.txt |
Wi-Fi regulatory country | |
/data/dragon.conf |
0x47ac88 | user settings (see default-dragon.conf) |
pud.date (per-unit data) |
### PerUnit parameters: ###, perunitData = { |
per-unit ISP parameters; INFERENCE: they come from the camera EEPROM |
/data/ftp/internal_000/Debug/... |
Alpaga/vision debug dumps |
/calib is read-only fixed data in the image (here populated). EVIDENCE of loader paths: /calib/custom_43_disto_lut.dat, /calib/DSL239-FThetaLUT.dat, /calib/SunnyMilos-FthetaLUT.dat, /calib/userTuning_impala.txt (the last logs loadUserTuningData(/calib/userTuning_impala.txt)). The other ftd_* and FThetaLUT_DSL217/DSL219 files have no literal path in dragon-prog (the paths are probably built from templates or consumed by the ISP library; INFERENCE) so their consumer is unconfirmed.
5. /calib contents
Section titled “5. /calib contents”| File | Content (EVIDENCE: read) |
|---|---|
| eeprom_cam.dat (8192 B) | Header MK3CAMERADATA 0x01 PI, serial placeholder 1234567890123456, date 20140905235959; at 0x30 u16 LE values 0x0900=2304, 0x066c=1644, 0x0780=1920 (2304x1644 = half of the 4608x3288 array, then 1920); float at 0x44 = 7.2978e-4 (unknown, possibly radians per pixel or scale); bytes 0x4c.. 00 40 00 00 40 40 40; at 0x80 ten u16 0x0100 (big-endian 1 or a flag array). Rest zero (last non-zero byte at 0xfdc). It looks like a generic/default image, not a real unit. |
| ftd_cc.txt | color correction: two rows, each <x> <y> <9 ints> (3x3 matrix, about 3700-4000 diagonal on a 4096 scale, two color temperatures/zones, e.g. `357 401 |
| ftd_curves.txt | 1024-line 10-bit curve, monotonic 0..~255 (looks like the gamma table) |
| ftd_dpc.txt | defective pixel list: 10 100 then triplets x y type (4608 x 3288 coordinate space, e.g. 3191 1806 1) |
| ftd_ee.txt | edge enhancement LUT, 6 lines of 256 values (ramps then clamps at 33) |
| ftd_nr.txt | noise reduction table, 14 columns x about 30 rows, values 0..110 |
| userTuning.txt, userTuning_impala.txt | small tuning blocks (second file adds -80 -80 -80 -64 -64 row = per-gain offsets; the “Impala” one is loaded) |
| DSL219/239/217, SunnyMilos F-theta, custom_43 disto | lens tables (section 2.4) |
6. Open questions
Section titled “6. Open questions”- Which
/dev/videoNis which: only string-adjacency evidence. A decompile of the V4L2 client open calls (HAL_v4l2_utils) would confirm. - The exact meaning of the last two floats in the 4 sub-entries of each video-mode record, and what produces 2112x1600/1344: need the consumer of the table (FUN_000960a0 callers around VIDEO_ThreadHandler) fully decompiled.
setCameraargument order (fx, fy, cx, cy, last=100.0) and what 100.0 is.- Where the condor vision output is consumed by the estimator and with which covariance (other agents’ scope).
- Which of DSL239/SunnyMilos/DSL219/DSL217 matches a Bebop 2 unit, and what the 7.2978e-4 float in the EEPROM is.
- Consumers of
ftd_cc/curves/dpc/ee/nr(no literal path found in dragon-prog; possibly in the statically linked Impala via a prefix). - The magic carpet detector location (not in libCondor exports).
- libdragonvideo/libsuperstream: used by which process (dragon-prog does not NEED them).
- H.264 profile/level/GOP/QP settings: the numbers are passed in
H264EncSetCodingCtrl/RateCtrlstructs built in dragon-prog; not decoded yet. - Whether the MT9V117 runs QVGA natively (320x240) or is scaled by the AVI; the vision context requests 320x240 UYVY.
Comments
Spotted a mistake, or tried this on your own Bebop? Leave a note. Comments are kept on this site's own server.