I would like to visualize pointcloud in drake-visualizer using python binding.
I imitated how to publish images through lcm from here, and checked out these two issues (14985, 14991). The snippet is as follows :
point_cloud_to_lcm_point_cloud = builder.AddSystem(PointCloudToLcm())
point_cloud_to_lcm_point_cloud.set_name('pointcloud_converter')
builder.Connect(
station.GetOutputPort('camera0_point_cloud'),
point_cloud_to_lcm_point_cloud.get_input_port()
)
point_cloud_lcm_publisher = builder.AddSystem(
LcmPublisherSystem.Make(
channel="DRAKE_POINT_CLOUD_camera0",
lcm_type=lcmt_point_cloud,
lcm=None,
publish_period=0.2,
# use_cpp_serializer=True
)
)
point_cloud_lcm_publisher.set_name('point_cloud_publisher')
builder.Connect(
point_cloud_to_lcm_point_cloud.get_output_port(),
point_cloud_lcm_publisher.get_input_port()
)
However, I got the following runtime error:
RuntimeError: DiagramBuilder::Connect: Mismatched value types while connecting output port lcmt_point_cloud of System pointcloud_converter (type drake::lcmt_point_cloud) to input port lcm_message of System point_cloud_publisher (type drake::pydrake::Object)
When I set 'use_cpp_serializer=True', the error becomes
LcmPublisherSystem.Make(
File "/opt/drake/lib/python3.8/site-packages/pydrake/systems/_lcm_extra.py", line 71, in _make_lcm_publisher
serializer = _Serializer_[lcm_type]()
File "/opt/drake/lib/python3.8/site-packages/pydrake/common/cpp_template.py", line 90, in __getitem__
return self.get_instantiation(param)[0]
File "/opt/drake/lib/python3.8/site-packages/pydrake/common/cpp_template.py", line 159, in get_instantiation
raise RuntimeError("Invalid instantiation: {}".format(
RuntimeError: Invalid instantiation: _Serializer_[lcmt_point_cloud]
I saw the cpp example here, so maybe this issue is specific to python binding.
I also saw this python example, but thought using 'PointCloudToLcm' might be more convenient.
P.S.
I am aware of the development in recent commits on MeshcatVisualizerCpp and MeshcatPointCloudVisualizerCpp, but I am still on the drake-dev stable build 0.35.0-1 and want to stay on drake visualizer until the meshcat c++ is more mature.
The old version in pydrake.systems.meshcat_visualizer.MeshcatVisualizer is a bit too slow on my current use-case (multiple objects drop). I can visualize the pointcloud with this visualization setting, but it took too much machine resources.
Only the message types that are specifically bound in lcm_py_bind_cpp_serializers.cc can be used on an LCM message input/output port connection between C++ and Python. For all other LCM message types, the input/output port connection must be from a Python system to a Python system or a C++ System to a C++ System.
The lcmt_image_array is listed there, but not the lcmt_point_cloud.
If you're stuck using Drake's v0.35.0 capabilities, then I don't see any great solutions. Some options:
(1) Write your own PointCloudToLcm system in Python (by re-working the C++ code into Python, possibly with a narrower set of supported features / channels for simplicity).
(2) Write your own small C++ helper function MakePointCloudPublisherSystem(...) that calls LcmPublisherSystem::Make<lcmt_point_cloud> function in C++, and bind it into Python. Then your Python code can call MakePointCloudPublisherSystem() and successfully connect that to the existing C++ PointCloudToLcm.
Related
I'm trying to write a simple machine learning application in Ada, and also trying to find a good framework to use. My knowledge of one thing is extremely minimal, and of the other is somewhat minimal.
There are several nifty machine learning frameworks out there, and I'd like to leverage one for use with an Ada program, but I guess I'm just...at a loss. Can I use an existing framework written in Python, for instance and wrap (or I guess, bind?) the API calls in Ada? Should I just pass off the scripting capabilities? I'm trying to figure it out.
Case in point: Scikit (sklearn)
https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html#
This does some neat stuff, and I'd like to be able to leverage this, but with an Ada program. Does anyone have advice from a similar experience?
I am just researching, so I have tried finding information.
http://www.inspirel.com/articles/Ada_Python_Binding.html
https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html#
The inspirel solution is based on python2.7. If you're using anything from python3.5 onwards a few mods need to be made. On Linux, changing to say python 3.7, you'd just change
--for Default_Switches ("Ada") use ("-lpython2.7");
for Default_Switches ("Ada") use ("-lpython3.7");
but on windows, the libraries aren't dumped in a community lib so gnat doesn't know where to find them. All the packages are kept separately. The -L has to be added to tell the linker where to find the library. Alternatively, you can use for lib_dir. In my case, I did a non-admin install of python, so it looks something like
for Default_Switches ("Ada") use ("-L\Users\StdUser\AppData\Local\Programs\Python\Python37-32\libs", "-lpython37");
Note that on windows, the library is called python37: not python3.7. Use gprbuild instead of gnatmake -p, which has been deprecated. If you do all your mods correctly
gprbuild ada_main.gpr
should give you an executable in obj\ada_main.exe if it builds. If a later version of python is used, some edits need to be made
python_module.py
#print 'Hello from Python module'
print('Hello from Python module')
#print 'Python adding:', a, '+', b
print('Python adding:', a, '+', b)
ada_main.adb
-- Python.Execute_String("print 'Hello from Python!'");
Python.Execute_String("print('Hello from Python!')");
Some routines have been deprecated so the linkage has to change
python.adb
--pragma Import(C, PyInt_AsLong, "PyInt_AsLong");
pragma Import(C, PyInt_AsLong, "PyLong_AsLong");
--pragma Import(C, PyString_FromString, PyString_FromString");
pragma Import(C, PyString_FromString, "PyUnicode_FromString");
Running the build and executable should give
C:\Users\StdUser\My Documents\ada-python>gprbuild ada_main.gpr
Compile
[Ada] ada_main.adb
Bind
[gprbind] ada_main.bexch
[Ada] ada_main.ali
Link
[link] ada_main.adb
C:\Users\StdUser\My Documents\ada-python>obj\ada_main.exe
executing Python directly from Ada:
Hello from Python!
loading external Python module and calling functions from that module:
Hello from Python module!
asking Python to add two integers:
Python adding: 10 + 2
Ada got result from Python: 12
we can try other operations, too:
subtract: 8
multiply: 20
divide : 5
Remember to put the pythonxx.dll somewhere on your path otherwise it won't be able to find the library when it starts executing.
I have an issue working with DiagramBuilder and ManipulationStation classes.
It appears to me, that c++ API and the python bindings work differently in my case.
C++ API behaves as expected, while the python bindings result in the runtime error:
DiagramBuilder: Cannot operate on ports of System plant until it has been registered using AddSystem
How I use C++ API
In one of the ManipulationStation::Setup...() methods I inject a block of code, that adds an extra manipuland
const std::string sdf_path = FindResourceOrThrow("drake/examples/manipulation_station/models/bolt_n_nut.sdf");
RigidTransform<double> X_WC(RotationMatrix<double>::Identity(), Vector3d(0.0, -0.3, 0.1));
bolt_n_nut_ = internal::AddAndWeldModelFrom(sdf_path, "nut_and_bolt", lant_->world_frame(), "bolt", X_WC, plant_);
I inject another block of code into the method ManipulationStation::Finalize:
auto zero_torque = builder.template AddSystem<systems::ConstantVectorSource<double>>(Eigen::VectorXd::Zero(plant_->num_velocities(bolt_n_nut_)));
builder.Connect(zero_torque->get_output_port(), plant_->get_actuation_input_port(bolt_n_nut_));
With these changes, the simulation runs as expected.
How I use python bindings
plant = station.get_multibody_plant()
manipuland_path = get_manipuland_resource_path()
bolt_with_nut = Parser(plant=plant).AddModelFromFile(manipuland_path)
X_WC = RigidTransform(RotationMatrix.Identity(), [0.0, -0.3, 0.1])
plant.WeldFrames(plant.world_frame(), plant.GetFrameByName('bolt', bolt_with_nut), X_WC)
...
station.Finalize()
zero_torque = builder.AddSystem(ConstantValueSource(AbstractValue.Make([0.])))
builder.Connect(zero_torque.get_output_port(), plant.get_actuation_input_port(bolt_with_nut_model))
This triggers a RuntimeError with a message as above; The port, which causes this error is nut_and_bolt_actuation.
My vague understanding of the problem is the (in) visibility of nut_and_bolt System, due to having two distinct DiagramBuilders in a process: 1) a one is inside ManipulationStation 2) another is in the python code, that instantiates this ManipulationStation object.
Using ManipulationStation via python bindings is a preference for me, because that way I would've avoided depending on a custom build of drake library.
Thanks for your insight!
I agree with your assessment: you have two different DiagramBuilder objects here. This does not have anything to due with C++ or Python; the ManipulationStation is itself a Diagram (created using its own DiagramBuilder), and you have a second DiagramBuilder (in either c++ or python) that is connecting the ManipulationStation together with other elements. You are trying to connect a system that is in the external diagram to a port that is in the internal diagram, but is not exposed.
The solution would be to have the ManipulationStation diagram expose the extra nut and bolt actuation port so that you can connect to it from the second builder.
If you prefer Python, I've switched my course to using a completely python version of the manipulation station. I find this version is much easier to adapt to different student projects. (To be clear, the setup is in python, but at simulation time all of the elements are c++ and it doesn't call back to python; so the performance is almost identical.)
I've been experimenting with PyCharm in conjunction with LO and UNO for some time (win10).
What I don't understand:
Why do I get the error Unresolved reference 'com' from PyCharm when I enter the code in editor, but everything seems to be OK, when I run the script with PyCharm or entered in console?
Example:
import uno
from com.sun.star.awt import Point
p = Point(100, 200)
print(p)
=> (com.sun.star.awt.Point){ X = (long)0x64, Y = (long)0xc8 }
What I would expect would be a relationship between the UNO interfaces com.sun.star.awt and Python so that I can use Intellisense/autocomplete (for example) when entering the code from com.sun.star.awt import Point in PyCharm'.
What I know is that all interfaces are described in the IDL files (SDK). I thought it must be possible to define a so-called stub/mock "package" that enables me to do this. Unfortunately, this request is beyond my capabilities at the moment. The attempt to integrate the idl structure from sdk is only inadequate (path structure only).
Are there any ready-made solutions? (which I doubt - intensively searched all sources)
Do I have to live with it now, or which approach would go in the right direction (I love intellisense)? - Maybe I think too complicated?
I'm trying to publish to the topic /joy/set_feedback.
I had this in my include:
#include <sensor_msgs/JoyFeedbackArray.h>
I had my Nodehandle and Publisher like this :
feed_pub = nh->advertise<sensor_msgs::JoyFeedbackArray>("/joy/set_feedback", 1);
Now my problem is that i want to fill up the feed_msg
sensor_msgs::JoyFeedbackArray feed_msg;
In the documentation sensor_msgs/JoyFeedbackArray Message
it says it's an array. No matter what i did wrote there, I always get an error.
It would be very helpful if i could get a correct example to fill up this array.
I'll be needing a vibration signal with type: 1 / id: 0 / intensity: 1.0.
ROS messages can handle something like Arrays. But when you are working with a ROS Message-"Array" in Python or C++, you need to use a different data type. In your case, you need a std::Vector<sensor_msgs::JoyFeedback>.
For other cases, the ROS Wiki has created a table of translations between ROS Message, C++ and Python:
You can find a detailed explaination of the different datatypes and their conversion between ROS Message and Python or C++ here: http://wiki.ros.org/msg
I am stumbling my way through writing a dissector for our custom protocol in Lua. While I have basic field extraction working, many of our fields have scale factors associated with them. I'd like to present the scaled value in addition to the raw extracted value.
It seems to me tree_item:add_packet_field is tailor-made for this purpose. Except I can't get it to work.
I found Mika's blog incredibly helpful, and followed his pattern for breaking my dissector into different files, etc. That's all working.
Given a packet type "my_packet", I have a 14-bit signed integer "AOA" that I can extract just fine
local pref = "my_packet"
local m = {
aoa = ProtoField.new("AOA", pref .. ".aoa", ftypes.INT16, nil, base.DEC, 0x3FFF, "angle of arrival measurement"),
}
local option=2
local aoa_scale = 0.1
function m.parse(tree_arg, buffer)
if option == 1 then
-- basic field extraction. This works just fine. The field is extracted and added to the tree
tree_arg:add(m.aoa, buffer)
elseif option == 2 then
-- This parses and runs. The item is decoded and added to the tree,
-- but the value of 'v' is always nil
local c,v = tree_arg:add_packet_field(m.aoa, buffer, ENC_BIG_ENDIAN)
-- this results in an error, doing arithmetic on 'nil'
c:append_text(" (scaled= " .. tostring(v*aoa_scale) .. ")")
end
end
(I use ProtoField.new instead of any of the type-specific variants for consistency in declaring my fields)
The documentation for add_packet_field says that the encoding argument is mandatory.
There is a README in the source code that says ENC_BIG_ENDIAN should be specified for network byte-order data (mine is). I know that section is for proto_tree_add_item, but I traced the code far enough to see that add_packet_field ends up passing the encoding to proto_tree_add_item.
Basically, at this point, I'm lost. I did find this post from 2014 that suggested limited support for add_packet_field but surely by now something as basic as an integer value is supported?
Also, I do know how to declare a Field and extract the value after tree:add does the parsing; worst case I'll fall back to that, but surely there is a more expedient way to access the just-parsed value added to the tree?
Wireshark Version
3.2.4 (v3.2.4-0-g893b5a5e1e3e)
Compiled (64-bit) with Qt 5.12.8, with WinPcap SDK (WpdPack) 4.1.2, with GLib
2.52.3, with zlib 1.2.11, with SMI 0.4.8, with c-ares 1.15.0, with Lua 5.2.4,
with GnuTLS 3.6.3 and PKCS #11 support, with Gcrypt 1.8.3, with MIT Kerberos,
with MaxMind DB resolver, with nghttp2 1.39.2, with brotli, with LZ4, with
Zstandard, with Snappy, with libxml2 2.9.9, with QtMultimedia, with automatic
updates using WinSparkle 0.5.7, with AirPcap, with SpeexDSP (using bundled
resampler), with SBC, with SpanDSP, with bcg729.
Running on 64-bit Windows 10 (1803), build 17134, with Intel(R) Xeon(R) CPU
E3-1505M v6 # 3.00GHz (with SSE4.2), with 32558 MB of physical memory, with
locale English_United States.1252, with light display mode, without HiDPI, with
Npcap version 0.9991, based on libpcap version 1.9.1, with GnuTLS 3.6.3, with
Gcrypt 1.8.3, with brotli 1.0.2, without AirPcap, binary plugins supported (19
loaded).
Built using Microsoft Visual Studio 2019 (VC++ 14.25, build 28614).
Looking at the try_add_packet_field() source code, only certain FT_ types are supported, namely:
FT_BYTES
FT_UINT_BYTES
FT_OID
FT_REL_OID
FT_SYSTEM_ID
FT_ABSOLUTE_TIME
FT_RELATIVE_TIME
None of the other FT_ types are supported [yet], including FT_UINT16, which is the one you're interested in here, i.e., anything else just needs to be done the old fashioned way.
If you'd like this to be implemented, I'd suggest filing a Wireshark enhancement bug request for this over at the Wireshark Bug Tracker.