Release v1.1 (#3)

FPGA Binaries v1.1 with support for AR1335 and OV5647 image sensors * support for AR1335, OV5647 image sensors * video creation script from FPGA Binary ISP output Burst Capture * Kria KV260 Binary file for use with AR1335 IAS image sensor module * Kria KV260 Binary file for use with OV5647 RPi image sensor module * header and description added * header and description added * header and description added * Create sensor_bin_to_sensor_raw_burst_capture.py * header updated * updated video_creation.py * video_creation.py updated * verified with SD card dumps * how to use release v1.1 binaries added --------- Co-authored-by: taimur-10xe <[email protected]> Co-authored-by: talhaiqbal-10xe <[email protected]> Co-authored-by: taimur-10xe <[email protected]>
10x-Engineers · Nov 27, 2023 · 0de929f · 0de929f
1 parent dd00e25
commit 0de929f
Show file tree

Hide file tree

Showing 8 changed files with 293 additions and 447 deletions.
diff --git a/README.md b/README.md
@@ -14,6 +14,8 @@ Infinite-ISP is a full-stack ISP development platform - from algorithm developme
 Infinite-ISP Image Signal Processing Pipeline FPGA binaries for XCK26 Zynq® UltraScale+™ MPSoC present on Xilinx® Kria™ KV260 Vision AI Starter Kit. Each binary file includes an FPGA bitstream paired with its firmware executable.
 
 # How to use the FPGA Binary Files
+
+## v1.0
 1. Connect the AR1335 IAS image sensor module (included in Xilinx Kria KV260 Accessory Pack) to the IAS1 port on the Kria KV260 AI Starter Kit.
 2. Insert an SD Card (2GB or above) into the Kria KV260 board.
 3. Connect the Kria KV260 board with a monitor screen using an HDMI cable.
@@ -23,6 +25,31 @@ Infinite-ISP Image Signal Processing Pipeline FPGA binaries for XCK26 Zynq® Ult
 7. Connect the USB cable with the JTAG/USB port on Kria to read the messages over UART.
 8. You can remove the SD card from the Kria board, extract the dumped single RGB-RAW pair and 300 RGB frames (if applicable), and visualize them using the provided scripts.
 
+## v1.1
+### For AR1335 IAS module:
+1. Connect AR1335 IAS image sensor module to the IAS1 port on Kria KV260 AI Starter Kit.
+2. Insert SD Card (2GB or above) into the Kria KV260 board.
+3. Connect the Kria KV260 board with a monitor screen using HDMI cable.
+4. Power up the Kria KV260 board and follow the [steps](https://docs.xilinx.com/r/en-US/ug1089-kv260-starter-kit/Ethernet-Recovery-Tool) for loading the on-board Xilinx Image Recovery Tool.
+5. Upload the desired binary file (e.g. Infinite-ISP_v1.1-AR1335-1080p.bin) provided in the release on the Kria KV260 AI Starter Kit.
+6. Reset the Kria KV260 board and visualize the Infinite-ISP output on your monitor screen.
+7. Connect the USB cable with the JTAG/USB port on Kria to read the messages over UART.
+8. RAW-ISPout image pair and Burst Capture frames dump in SD Card status will be displayed over UART.
+9. Once SD Card dumps are complete, you can remove the SD card from the Kria board.
+10. Extract the dumped single RAW-ISPout pair and Burst Capture frames from the SD card and visualize them using provided scripts.
+
+### For OV5647 image sensor module:
+1. Connect OV5647 image sensor module to the RPi port on Kria KV260 AI Starter Kit.
+2. Insert SD Card (2GB or above) into the Kria KV260 board.
+3. Connect the Kria KV260 board with a monitor screen using HDMI cable.
+4. Power up the Kria KV260 board and follow the [steps](https://docs.xilinx.com/r/en-US/ug1089-kv260-starter-kit/Ethernet-Recovery-Tool) for loading the on-board Xilinx Image Recovery Tool.
+5. Upload the desired binary file (e.g. Infinite-ISP_v1.1-OV5647-1080p.bin) provided in the release on the Kria KV260 AI Starter Kit.
+6. Reset the Kria KV260 board and visualize the Infinite-ISP output on your monitor screen.
+7. Connect the USB cable with the JTAG/USB port on Kria to read the messages over UART.
+8. RAW-ISPout image pair and Burst Capture frames dump in SD Card status will be displayed over UART.
+9. Once SD Card dumps are complete, you can remove the SD card from the Kria board.
+10. Extract the dumped single RAW-ISPout pair and Burst Capture frames from the SD card and visualize them using provided scripts.
+
 ## Contact
 For any inquiries or feedback, feel free to reach out.
 

diff --git a/binaries/Infinite-ISP_v1.1-AR1335-1080p.bin b/binaries/Infinite-ISP_v1.1-AR1335-1080p.bin
diff --git a/binaries/Infinite-ISP_v1.1-OV5647-1080p.bin b/binaries/Infinite-ISP_v1.1-OV5647-1080p.bin
diff --git a/scripts/isp_output_bin_to_isp_output_png.py b/scripts/isp_output_bin_to_isp_output_png.py
@@ -1,16 +1,26 @@
+"""
+File: isp_output_bin_to_isp_output_png.py
+Description: converts the ISP output memory dump (.bin) data
+             from the FPGA Platform to output image frame (.png) 
+             as well as output pixel data frame (.bin) for
+             verification
+Author: 10xEngineers
+------------------------------------------------------------
+"""
+
 import numpy as np
 import matplotlib.image as mpimg
 import matplotlib.pyplot as plt
 from datetime import datetime
 
 
 filepath = "./"
-filename = 'RTL_ImageTest_2048x1536_10bit_GRBG_1920x1080.bin'
+filename = 'ISPOut_TestImage_2592x1944_10bits_BGGR.bin'
 with open(filepath + filename, 'rb') as f:
     arr = np.fromfile(f, dtype=np.uint8)
 f.close()
 
-h, w, Format, CONV_STD = 1080, 1920, "BGR", 2
+h, w, Format, CONV_STD = 1080, 1920, "YUV444", 2   #copy string from support output formats here
 
 SupportedFormats = {
     "BGR"   : 1,

diff --git a/scripts/sensor_bin_to_sensor_raw.py b/scripts/sensor_bin_to_sensor_raw.py
@@ -1,25 +1,27 @@
-# -*- coding: utf-8 -*-
 """
-Created on Tue Feb 21 16:07:11 2023
-
-@author: user3
+File: sensor_bin_to_sensor_raw.py
+Description: converts the image sensor memory dump (.bin) data
+             from the FPGA Platform to Bayer RAW frame (.raw) 
+             containing valid pixel data
+Author: 10xEngineers
+------------------------------------------------------------
 """
+
 import numpy as np
-import cv2
 import matplotlib.image as mpimg
 import matplotlib.pyplot as plt
 
-# Supported Sensors
-
-SENSOR = "AR1335"
+# Sensor selection for conversion to .raw
+SENSOR = "OV5647"   #copy string from supported sensor type here
 
+# Supported Sensors
 SupportedSensors = {
     "AR1335": 1,
     "OV5647": 2 
 }
 
 # reading the dumped binary file
-filename = 'ImageTest_2048x1536_10bit_GRBG_1920x1080.bin'
+filename = 'RAW_TestImage_2592x1944_10bits_BGGR.bin'
 with open(filename, 'rb') as f:
     # read the contents of the file into a new array
     arr = np.fromfile(f, dtype=np.uint8)

diff --git a/scripts/sensor_bin_to_sensor_raw_burst_capture.py b/scripts/sensor_bin_to_sensor_raw_burst_capture.py
@@ -0,0 +1,90 @@
+"""
+File: sensor_bin_to_sensor_raw_burst_capture.py
+Description: converts the image sensor memory dumps (.bin) of
+             RAW Burst Capture from the FPGA Platform to 
+             Bayer RAW frames (.raw) containing valid pixel
+             data.
+             It also converts the Bayer RAW frames to
+             equivalent grayscale .png for visualization.
+Author: 10xEngineers
+------------------------------------------------------------
+"""
+import numpy as np
+import matplotlib.image as mpimg
+import matplotlib.pyplot as plt
+from pathlib import Path
+
+# Path of the directory containing "OV5467" and "AR1335" directories
+path =  "./"
+
+# Supported Sensors and selected sensor (SENSOR)
+SupportedSensors = {
+    "AR1335": 1,
+    "OV5647": 2 
+}
+SENSOR = "AR1335"
+
+# start and end index of burst capture frames for converting .bin to .raw
+start, end = 1, 250
+
+# parent directory
+p = Path(path)
+parent_dir = p.resolve().joinpath(SENSOR)   
+
+# Making directories for saving .raw and .png files
+raw_dir = "Burst_Capture_RAW"
+png_dir = "Burst_Capture_PNG"
+bin_dir = "Burst_Capture"       # Created by firmware, DO NOT MODIFY
+# joining paths with parent directory
+raw_path = parent_dir/ raw_dir 
+Path.mkdir(raw_path, exist_ok=True)
+png_path = parent_dir / png_dir
+Path.mkdir(png_path, exist_ok=True)
+bin_path = parent_dir / bin_dir
+
+# Selecting height and width based on selected sensor
+if(SupportedSensors[SENSOR] == SupportedSensors["AR1335"]):
+    h, w = 1536, 2048
+
+if(SupportedSensors[SENSOR] == SupportedSensors["OV5647"]):
+    h, w = 1944, 2592
+
+h =  int(h)
+w =  int(w)
+
+# Images are stored in the form of rows where the size of each row in bytes
+# should be a multiple of 256, each such row size is called 'stride'
+# For raw10 format, 3 pixels are packed into 4 bytes
+stride = np.floor(np.floor(np.floor((w+2)/3) *4 +256 - 1) /256) * 256
+stride = stride.astype (np.uint16)
+pixelsInStride= int((stride/4)*3)
+
+for index in range (start, end+1):
+# reading the dumped binary file
+    filename = 'RAW' + str(index) + '.bin'
+    filepath = bin_path / filename
+    print('Processing ' + 'RAW'+ str(index) + ' ...')
+    with open(filepath, 'rb') as f:
+        # read the contents of the file into a new array
+        arr = np.fromfile(f, dtype=np.uint8)
+
+    # Reshape the array into groups of 4 elements
+    grouped_array = arr.reshape(-1, 4)
+    flipped_array = np.flip(grouped_array, axis=1)
+    result_list = []
+    for inner_array in flipped_array:
+        # Convert each element to binary and concatenate
+        binary_concatenated = ''.join([format(x, '08b') for x in inner_array])
+        result_list.append((int(binary_concatenated[2:12],2),int(binary_concatenated[12:22],2),int(binary_concatenated[22:32],2)))
+    img = np.array(result_list).reshape((h, pixelsInStride))[:, 0:w].astype(np.uint16)
+
+    # dumping a .raw file for inf_isp
+    filename = filename[:-4]
+    extension = ".raw"
+
+    with open('{}{}'.format(str(raw_path/filename),extension),'wb') as f:
+        img.tofile(f)
+
+    # dumping a numpy array
+    img_norm = np.interp(img, (img.min(), img.max()), (0,1023 )).astype(np.uint8)
+    plt.imshow(img_norm,cmap='gray').write_png(str(png_path/filename) + '.png')