AXI stream interfaces in Xilinx system generator IP - image-processing

I have an example design in system generator for image processing which has one input image and one output image.
I would like to send data through AXI stream interface and export it as an IP core to Vivado IP integrator and develop the design further using DMA and software in SDK.
Firstly is it possible to have AXI stream interface in my design? If yes, how can I implement it? Can anybody help me?
Thanks in advance.
(i have attached image of the example here)
Image_filter

Firstly is it possible to have AXI stream interface in my design?
Yes, it is.
If yes, how can I implement it? Can anybody help me?
I have a similar project develop in Vivado 2015.3: an image filter (created with "High Level Synthesis") and this design block:
The High Level Synthesis code should look like:
#include "top.h"
void hls_sobel(
hls::stream< ap_axiu<8,1,1,1> > &video_in,
hls::stream< ap_axiu<8,1,1,1> > &video_out
)
{
ap_uint<16> Image_w=IMAGE_W_MAX;
ap_uint<16> Image_h=IMAGE_H_MAX;
// Create AXI streaming interfaces for the core
#pragma HLS INTERFACE axis port=video_in bundle=video_in
#pragma HLS INTERFACE axis port=video_out bundle=video_out
// No control interface - auto-start as soon as there's an input frame
#pragma HLS INTERFACE ap_ctrl_none port=return // no handshakes
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_8UC1> mat_in(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_8UC1> mat_out(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_8UC1> inx(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_8UC1> iny(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_16SC1> sobelx(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_16SC1> sobely(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_16SC1> zerox(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_16SC1> zeroy(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_8UC1> absx(Image_h, Image_w);
hls::Mat<IMAGE_H_MAX, IMAGE_W_MAX, HLS_8UC1> absy(Image_h, Image_w);
#pragma HLS dataflow
// read input and convert from axi-stream to Mat
hls::AXIvideo2Mat(video_in, mat_in);
// calculate Sobel in X and Y directions
hls::Duplicate(mat_in, inx, iny);
hls::Sobel<1,0,3>(inx, sobelx);
hls::Sobel<0,1,3>(iny, sobely);
// calculate abs of said Sobel
hls::Zero(zerox);
hls::Zero(zeroy);
hls::AbsDiff(sobelx, zerox, absx);
hls::AbsDiff(sobely, zeroy, absy);
// add both abs
hls::AddWeighted(absx, 1, absy, 1, 0, mat_out);
// write output
hls::Mat2AXIvideo(mat_out, video_out);
}
As you can note, a DMA is used. For the Video-Image application, I recommend using a Video-DMA (VDMA) to send all the pixel information via streaming-interface. After, in the SDK, it is easy to manage the transfer using the function in the Board Support Package (BSP).
Also, you can note that in the code above is explicitly specified hls::stream< ap_axiu<8,1,1,1> > &video_in,! In this way, I am creating a streaming interface.
Here you can find a tutorial about HLS image processing filter. In the last page, there are useful links. Follow them to realize the same system of the example.
I hope this can help

I am currently working on a very similar project, (I am not using System Generator though), so I bet I could give you some pointers. As far as I remember SysGen can produce some VHDL or Verilog code of your design. So:
After you get the HDL code of your design, pack it as a new IP in Vivado. There are plenty of tutorials on how to do this, it should be easy with a little search. You should wisely choose the interfaces you want to implement. You're going to definitely need an AXI Stream Slave interface for accepting the incoming data and an AXI Stream Master interface to transmit the results.
After you package your IP, you can begin building your system block-by-block (there are also some good tutorials on this, see end of answer). You will need to use the AXI DMA IP (or the Video DMA, depending on your needs) and you'll have to configure it properly, like choosing register-mode or scatter-gather, channels, etc.
Be extremely careful to generate the proper AXI synchronization signals correctly, as they can totally ruin your design (and nerves). It's easy but it requires some study of the AXI documentation provided by Xilinx (ARM's docs are too complicated for my taste).
Finally, you will definitely find very useful information on the following resources:
Xilinx Forum
FPGAdeveloper's example
another AXI-stream based design example
FPGA note wiki
AXI DMA Product Guide
Channel of Dr. Sadri of TU Kaiserslautern, really helpful to deeply understand AXI design concepts
Good luck!
PS: Simulators are your friends! Never try implementing your freshly written code directly onto the system design. Modelsim can save you significant time and effort which would otherwise be spent on pointless debugging.

If I understand correctly, you want to know how to create an AXI Stream interface inside your system generator design.
Yes it is possible to do it. You should have atleast two inputs in your design with names, for example, image_tdata and image_tvalid (gateway in). When you generate IP core, sysgen will recoginize this as an AXI STREAM. The format is important. It must be "$customname_tdata" and "$customname_tvalid". You can add other inputs as well to add to the AXI STREAM such as "$customname_tlast", "$customname_tready".

Related

DiagramBuilder: Cannot operate on ports of System plant until it has been registered using AddSystem

I have an issue working with DiagramBuilder and ManipulationStation classes.
It appears to me, that c++ API and the python bindings work differently in my case.
C++ API behaves as expected, while the python bindings result in the runtime error:
DiagramBuilder: Cannot operate on ports of System plant until it has been registered using AddSystem
How I use C++ API
In one of the ManipulationStation::Setup...() methods I inject a block of code, that adds an extra manipuland
const std::string sdf_path = FindResourceOrThrow("drake/examples/manipulation_station/models/bolt_n_nut.sdf");
RigidTransform<double> X_WC(RotationMatrix<double>::Identity(), Vector3d(0.0, -0.3, 0.1));
bolt_n_nut_ = internal::AddAndWeldModelFrom(sdf_path, "nut_and_bolt", lant_->world_frame(), "bolt", X_WC, plant_);
I inject another block of code into the method ManipulationStation::Finalize:
auto zero_torque = builder.template AddSystem<systems::ConstantVectorSource<double>>(Eigen::VectorXd::Zero(plant_->num_velocities(bolt_n_nut_)));
builder.Connect(zero_torque->get_output_port(), plant_->get_actuation_input_port(bolt_n_nut_));
With these changes, the simulation runs as expected.
How I use python bindings
plant = station.get_multibody_plant()
manipuland_path = get_manipuland_resource_path()
bolt_with_nut = Parser(plant=plant).AddModelFromFile(manipuland_path)
X_WC = RigidTransform(RotationMatrix.Identity(), [0.0, -0.3, 0.1])
plant.WeldFrames(plant.world_frame(), plant.GetFrameByName('bolt', bolt_with_nut), X_WC)
...
station.Finalize()
zero_torque = builder.AddSystem(ConstantValueSource(AbstractValue.Make([0.])))
builder.Connect(zero_torque.get_output_port(), plant.get_actuation_input_port(bolt_with_nut_model))
This triggers a RuntimeError with a message as above; The port, which causes this error is nut_and_bolt_actuation.
My vague understanding of the problem is the (in) visibility of nut_and_bolt System, due to having two distinct DiagramBuilders in a process: 1) a one is inside ManipulationStation 2) another is in the python code, that instantiates this ManipulationStation object.
Using ManipulationStation via python bindings is a preference for me, because that way I would've avoided depending on a custom build of drake library.
Thanks for your insight!
I agree with your assessment: you have two different DiagramBuilder objects here. This does not have anything to due with C++ or Python; the ManipulationStation is itself a Diagram (created using its own DiagramBuilder), and you have a second DiagramBuilder (in either c++ or python) that is connecting the ManipulationStation together with other elements. You are trying to connect a system that is in the external diagram to a port that is in the internal diagram, but is not exposed.
The solution would be to have the ManipulationStation diagram expose the extra nut and bolt actuation port so that you can connect to it from the second builder.
If you prefer Python, I've switched my course to using a completely python version of the manipulation station. I find this version is much easier to adapt to different student projects. (To be clear, the setup is in python, but at simulation time all of the elements are c++ and it doesn't call back to python; so the performance is almost identical.)

Pointcloud Visualization in Drake Visualizer in Python

I would like to visualize pointcloud in drake-visualizer using python binding.
I imitated how to publish images through lcm from here, and checked out these two issues (14985, 14991). The snippet is as follows :
point_cloud_to_lcm_point_cloud = builder.AddSystem(PointCloudToLcm())
point_cloud_to_lcm_point_cloud.set_name('pointcloud_converter')
builder.Connect(
station.GetOutputPort('camera0_point_cloud'),
point_cloud_to_lcm_point_cloud.get_input_port()
)
point_cloud_lcm_publisher = builder.AddSystem(
LcmPublisherSystem.Make(
channel="DRAKE_POINT_CLOUD_camera0",
lcm_type=lcmt_point_cloud,
lcm=None,
publish_period=0.2,
# use_cpp_serializer=True
)
)
point_cloud_lcm_publisher.set_name('point_cloud_publisher')
builder.Connect(
point_cloud_to_lcm_point_cloud.get_output_port(),
point_cloud_lcm_publisher.get_input_port()
)
However, I got the following runtime error:
RuntimeError: DiagramBuilder::Connect: Mismatched value types while connecting output port lcmt_point_cloud of System pointcloud_converter (type drake::lcmt_point_cloud) to input port lcm_message of System point_cloud_publisher (type drake::pydrake::Object)
When I set 'use_cpp_serializer=True', the error becomes
LcmPublisherSystem.Make(
File "/opt/drake/lib/python3.8/site-packages/pydrake/systems/_lcm_extra.py", line 71, in _make_lcm_publisher
serializer = _Serializer_[lcm_type]()
File "/opt/drake/lib/python3.8/site-packages/pydrake/common/cpp_template.py", line 90, in __getitem__
return self.get_instantiation(param)[0]
File "/opt/drake/lib/python3.8/site-packages/pydrake/common/cpp_template.py", line 159, in get_instantiation
raise RuntimeError("Invalid instantiation: {}".format(
RuntimeError: Invalid instantiation: _Serializer_[lcmt_point_cloud]
I saw the cpp example here, so maybe this issue is specific to python binding.
I also saw this python example, but thought using 'PointCloudToLcm' might be more convenient.
P.S.
I am aware of the development in recent commits on MeshcatVisualizerCpp and MeshcatPointCloudVisualizerCpp, but I am still on the drake-dev stable build 0.35.0-1 and want to stay on drake visualizer until the meshcat c++ is more mature.
The old version in pydrake.systems.meshcat_visualizer.MeshcatVisualizer is a bit too slow on my current use-case (multiple objects drop). I can visualize the pointcloud with this visualization setting, but it took too much machine resources.
Only the message types that are specifically bound in lcm_py_bind_cpp_serializers.cc can be used on an LCM message input/output port connection between C++ and Python. For all other LCM message types, the input/output port connection must be from a Python system to a Python system or a C++ System to a C++ System.
The lcmt_image_array is listed there, but not the lcmt_point_cloud.
If you're stuck using Drake's v0.35.0 capabilities, then I don't see any great solutions. Some options:
(1) Write your own PointCloudToLcm system in Python (by re-working the C++ code into Python, possibly with a narrower set of supported features / channels for simplicity).
(2) Write your own small C++ helper function MakePointCloudPublisherSystem(...) that calls LcmPublisherSystem::Make<lcmt_point_cloud> function in C++, and bind it into Python. Then your Python code can call MakePointCloudPublisherSystem() and successfully connect that to the existing C++ PointCloudToLcm.

ROS - How do I publish a message and get the subscribed callback immediately

I have a ROS node that allows you to "publish" a data structure to it, to which it responds by publishing an output. The timestamp of what I published and what it publishes is matched.
Is there a mechanism for a blocking function where I send/publish and output, and it waits until I receive an output?
I think you need the ROS_Services (client/server) pattern instead of the publisher/subscriber.
Here is a simple example to do that in Python:
Client code snippet:
import rospy
from test_service.srv import MySrvFile
rospy.wait_for_service('a_topic')
try:
send_hi = rospy.ServiceProxy('a_topic', MySrvFile)
print('Client: Hi, do you hear me?')
resp = send_hi('Hi, do you hear me?')
print("Server: {}".format(resp.response))
except rospy.ServiceException, e:
print("Service call failed: %s"%e)
Server code snippet:
import rospy
from test_service.srv import MySrvFile, MySrvFileResponse
def callback_function(req):
print(req)
return MySrvFileResponse('Hello client, your message received.')
rospy.init_node('server')
rospy.Service('a_topic', MySrvFile, callback_function)
rospy.spin()
MySrvFile.srv
string request
---
string response
Server out:
request: "Hi, do you hear me?"
Client out:
Client: Hi, do you hear me?
Server: Hello client, your message received.
Learn more in ros-wiki
Project repo on GitHub.
[UPDATE]
If you are looking for fast communication, TCP-ROS communication is not your purpose because it is slower than a broker-less communicator like ZeroMQ (it has low latency and high throughput):
ROS-Service pattern equivalent in ZeroMQ is REQ/REP (client/server)
ROS publisher/subscriber pattern equivalent in ZeroMQ is PUB/SUB
ROS publisher/subscriber with waitformessage equivalent in ZeroMQ is PUSH/PULL
ZeroMQ is available in both Python and C++
Also, to transfer huge amounts of data (e.g. pointcloud), there is a mechanism in ROS called nodelet which is supported only in C++. This communication is based on shared memory on a machine instead of TCP-ROS socket.
What exactly is a nodelet?
Since you want to stick with publish/ subscribers, assuming from your comment, that services are to slow I would have a look at waitForMessage (Documentation).
And for an example on how to use it you can have a look at this ros answers question.
All you need to do is to publish your data and immediately call waitForMessage on the output topic and manually pass the received message to your "callback".
I hope this is what you were looking for.
To get this request/reply behaviour ROS has a mechanism called ROS service.
You can specify the input and output of your service in a service file similar to a ROS message definition. You can then call the service of a node with your input and the call will receive an output when the service is finished.
Here is a tutorial how to use this mechanism in python. If you prefer C++ there is also one, you should find it.

how to write driver for MX 6 and TJA1100 PHY?

I am working on new costume board based on (i. MX 6Solo6DualLite).and I use (TJA1100 100BASE-T1 PHY) for Automotive Ethernet..
please correct me if my questions don't make sense, or I am in the wrong way.
I want to write driver for this device.. and make sure that it works correctly.
1- I can make sure that the driver works properly in the U-boot step, Right? I mean no need to load Linux kernel, so I have just to add source code C driver in U-boot source code and compile it. I want to do this in U-boot step, so I can limit the numbers of files that initialise all peripheriques, and make it simple as possible since that U-boot can behave like (mini-Os)
2-I don't know how to write this driver (exactly..), so I am looking for the driver (source code) that initialise the Ethernet Controller in any other processor , and initialise another typeof ethernet phy, in order to get an idea and write a similar driver source code for I.MX6 and TJA1100,?
after this i think that i could maybe add some very basic file c For simple Protocol like ARP, for test purpose..
3- is this good idea writing driver code by inspiring from another driver code source?
4 - maybe, if you already have a driver for (i. MX 6Solo6DualLite and TJA1100 100BASE-T1 PHY) can you provide to me please... ?
for my second question i tried to extract from U-boot source code the C file that initialise Ethernet Controller in AM335x, and initialise LAN8710A phy,(in beaglebone black) in order to get an idea and write a similar driver source code but i couldn't found it .. i found network C file for protocol .... but that's disturp me i couldn't seperate them from the real C file that initialise ethernet controller and ethernent Phy .
http://www.denx.de/wiki/U-Boot/SourceCode
There is a driver published on the NXP forum:
https://community.nxp.com/thread/460767
It includes both some bare metal code that should be usable with U-Boot and a Linux driver.

Stream reasoning / Reactive programming in prolog?

I was wondering if you know of any way to use prolog for stream processing, that is, some kind of reactive programming, or at least to let a query run on a knowledge base that is continuously updated (effectively a stream), and continuously output the output of the reasoning?
Anything implemented in the popular "prologs", such as SWI-prolog?
You can use Logtalk's support for event-driven programming to define monitors that watch for knowledge base update events and react accordingly. You can run Logtalk using most Prolog systems as the backed compiler, including SWI-Prolog.
The event-driven features are described e.g. in the user manual:
http://logtalk.org/manuals/userman/events.html
The current distribution contains some examples of using events and monitors. An interesting one considering your question is the bricks example:
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/bricks
Running this example first and then looking at its code should give you as good idea of what you can do with system wide events and monitors.
XSB has stream processing capabilities. See page 14 in the
XSB Manual
I'm working on something related, in project pqConsole already there is the basic capability: report to the user structured data, containing actionable areas (links) that call back in Prolog current state, hence the possibility to expose actions and react appropriately (hopefully).
It's strictly related to pqConsole::win_write_html, showcasing recent Qt capabilities of SWI-Prolog.
Here an example of a snippet producing only a simple formatted report, I'll try now to add the reactive part, so you can evaluate if you find expressive this basic system. Hints are welcome...
/* File: win_html_write_test.pl
Author: Carlo,,,
Created: Aug 27 2013
Purpose: example usage win_html_write/1
*/
:- module(win_html_write_test,
[dir2list/1
]).
:- [library(http/html_write)].
:- [library(dirtree)].
dir2list(Path) :-
dirtree(Path, DirTree),
% sortree(compare_by_attr(name), DirTree, Sorted), !,
DirTree = Sorted,
phrase(html([\css,
\logo,
hr([]),
ul(\dirtree2html(Sorted, [])),
br([])]), Tokens),
with_output_to(atom(X), print_html(Tokens)),
win_html_write(X),
dump_page_to_debug(X).
css --> html(style(type='text/css',
['.size{color:blue;}'
])).
logo --> html(img([src=':/swipl.png'],[])).
dirtree2html(element(dir, A, S), Parents) -->
html(li([\elem2html(A),
ul(\elements2html(S, [A|Parents]))])).
dirtree2html(element(file, A, []), _Parents) -->
html(li(\elem2html(A))).
elem2html(A) -->
{memberchk(name=N, A),
memberchk(size=S, A)
},
html([span([class=name], N), ' : ', span([class=size], S)]).
elements2html([E|Es], Parents) -->
dirtree2html(E, Parents),
elements2html(Es, Parents).
elements2html([], _Parents) --> [].
dump_page_to_debug(X) :-
open('page_to_debug.html', write, S),
format(S, '<html>~n~s~n</html>~n', [X]),
close(S).
This snippet depends on dirtree, that should be installed with
?- pack_install(dirtree).
edit With 3 modifications now the report has the ability to invoke editing of files:
call to get paths in structure
dir2list(Path) :-
dirtree(Path, DirTreeT),
assign_path(DirTreeT, DirTree),
...
request a specialized output for files only
dirtree2html(element(file, A, []), _Parents) -->
html(li(\file2html(A))).
finally, the 'handler' - here just place a request to invoke the editor
file2html(A) -->
{memberchk(name=N, A),
memberchk(path=P, A),
memberchk(size=S, A)
},
html([span([class=name],
[a([href='writeln(editing(\'~s\')), edit(\'~s\')'-[N,P]], N)]
), ' : ', span([class=size], S)]).
and now the file names are clickable, write a message and get edited if requested: a picture
You should check out RTEC: Run-Time Event Calculus:
https://github.com/aartikis/RTEC
RTEC is an open-source Event Calculus dialect optimised for stream reasoning. It is written in Prolog and has been tested under YAP 6.2.
Feature highlights:
Interval-based.
Sliding window reasoning.
Interval manipulation constructs for non-inertial fluents.
Caching for hierarchical knowledge bases.
Support for out-of-order data streams.
Indexing for handling efficiently irrelevant data.
There is also a mention of it on the SWI-Prolog website:
https://www.swi-prolog.org/pack/file_details/prologmud_I7/prolog/ec_planner/RTEC/README.md
which presumably relies on:
https://www.swi-prolog.org/pldoc/doc/_SWI_/library/dialect/yap.pl
I don't know why this hasn't been brought up so far, but in SWI-Prolog there is prolog_listen, which can, amongst other things, monitor dynamic updates to the database:

Resources