Convert pcap to csv: CICFlowmeter throwing - scapy.error.Scapy_Exception: tcpdump is not available error when trying to convert pcap file to csv - wireshark

I am following the steps as mentioned in https://pypi.org/project/cicflowmeter/ to convert a pcap to csv cicflowmeter -f example.pcap -c flows.csv and this error is thrown
(pcap) C:\Users\learner3\Anaconda3\envs\pcap\Scripts>cicflowmeter.exe -f test_pcap.pcap -c test_pcap.csv
WARNING: WinPcap is now deprecated (not maintained). Please use Npcap instead
Exception in thread AsyncSniffer:
Traceback (most recent call last):
File "C:\Users\learner3\Anaconda3\envs\pcap\lib\threading.py", line 980, in _bootstrap_inner
self.run()
File "C:\Users\learner3\Anaconda3\envs\pcap\lib\threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\learner3\Anaconda3\envs\pcap\lib\site-packages\scapy\sendrecv.py", line 1072, in _run
sniff_sockets.update((PcapReader(
File "C:\Users\learner3\Anaconda3\envs\pcap\lib\site-packages\scapy\sendrecv.py", line 1074, in <genexpr>
tcpdump(fname,
File "C:\Users\learner3\Anaconda3\envs\pcap\lib\site-packages\scapy\utils.py", line 2095, in tcpdump
raise Scapy_Exception(
scapy.error.Scapy_Exception: tcpdump is not available
However the UI of CICFLOWmeter worked fine and the pcap was converted to csv.
How can I fix this?
Any other approach using python code will also be very helpful, though I want all the fields that are specified here https://github.com/ahlashkari/CICFlowMeter/blob/master/ReadMe.txt
I actually want to be able to do this inside a linux/ubuntu VM, that's why the UI approach wont be feasible, hence I was trying with the command based approach, that gave the above error.
Any help is greatly appreciated!

Related

Ros2 not finding executables running in Docker

I'm trying to run a python script with ros2 in my docker container, and everything up to running the Script works, I can even run Gazebo via a launch file, and it works.
The Error ROS gives me is the following:
root#86d8bf3a6eb9:/# ros2 run field_robot robot_spawner.py
Traceback (most recent call last):
File "/opt/ros/foxy/bin/ros2", line 11, in <module>
load_entry_point('ros2cli==0.9.11', 'console_scripts', 'ros2')()
File "/opt/ros/foxy/lib/python3.8/site-packages/ros2cli/cli.py", line 67, in main
rc = extension.main(parser=parser, args=args)
File "/opt/ros/foxy/lib/python3.8/site-packages/ros2run/command/run.py", line 70, in main
return run_executable(path=path, argv=args.argv, prefix=prefix)
File "/opt/ros/foxy/lib/python3.8/site-packages/ros2run/api/__init__.py", line 61, in run_executable
process = subprocess.Popen(cmd)
File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/field_robot/dev_ws/install/field_robot/lib/field_robot/robot_spawner.py'
And yes, I checked, the File actually exists:
root#86d8bf3a6eb9:/# ls -l /field_robot/dev_ws/install/field_robot/lib/field_robot/robot_spawner.py
-rwxr-xr-x 1 root root 1964 Apr 12 14:37 /field_robot/dev_ws/install/field_robot/lib/field_robot/robot_spawner.py
Also, I'm running the Host system on Windows, so it could be that something with windows is fucked up, so if you have an Idea what could be the Problem there, that also might be it
Based on the comments it appears you're running into this issue because of the file type. If they're being edited in Windows first it is likely they are DOS files and not UNIX files. I know this causes issues with ROS1 so I assume it's the case in ROS2 as well. To fix this, you have a couple of options.
Usually the easiest would be to use dos2unix. This isn't installed by default but you can get it via apt install dos2unix assuming your image is Ubuntu. The files can be converted by running dos2unix <filename> inside your container.

JupyterHub - oauth_client_id not found

I am using Azure to run python notebook using Jupyterhub. After spinning up the VM, I was able to access the notebooks just by using my username and password (just like ssh). However, one day later when I switched to another network (I am not claiming that the network might have been a problem) I am unable to access the link. It gives me The site can't be reached error.
So I tried rerunning the process again, and since then I have been struggling to make it run again. I have searched for similar issues on GitHub, but they aren't helpful either.
After the kill the process using kill pid command, I tried running the jupyterhub through this command:
/anaconda/envs/py35/bin/python /anaconda/envs/py35/bin/jupyterhub-singleuser --port=50387 --notebook-dir="~/notebooks" --config=/etc/jupyterhub/jupyterhub_config.py
And it gives me the error:
JUPYTERHUB_API_TOKEN env is required to run jupyterhub-singleuser. Did you launch it manually?
So I searched through github issues similar to this. I tried generating token manually using:
jupyterhub token username
And I added that token to JUPYTERHUB_API_TOKEN via export JUPYTERHUB_API_TOKEN=token. I also added token:username to c.Authenticator.tokens in jupyterhub_config.py. Now I get this error:
Traceback (most recent call last):
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 528, in get
value = obj._trait_values[self.name]
KeyError: 'oauth_client_id'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/py35/bin/jupyterhub-singleuser", line 6, in <module>
main()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 455, in main
return SingleUserNotebookApp.launch_instance(argv)
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/notebook/notebookapp.py", line 1296, in initialize
self.init_webapp()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 393, in init_webapp
self.init_hub_auth()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 388, in init_hub_auth
if not self.hub_auth.oauth_client_id:
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 556, in __get__
return self.get(obj, cls)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 535, in get
value = self._validate(obj, dynamic_default())
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 593, in _validate
value = self._cross_validate(obj, value)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 599, in _cross_validate
value = obj._trait_validators[self.name](obj, proposal)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 907, in __call__
return self.func(*args, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/services/auth.py", line 439, in _ensure_not_empty
raise ValueError("%s cannot be empty." % proposal.trait.name)
ValueError: oauth_client_id cannot be empty.
I am not sure where I went wrong in this process. Anybody familiar with this issue?
Try running jupyterhub instead of jupyterhub-singleuser
For your specific use case, the command would be as follows:
sudo /anaconda/envs/py35/bin/python /anaconda/envs/py35/bin/jupyterhub --port=50387 --notebook-dir="~/notebooks" --config=/etc/jupyterhub/jupyterhub_config.py
Make sure that jupyterhub is installed (correctly) in the path you mentioned.

apache_beam.runners.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed:

I set up a Google Cloud project in Cloud Shell, and tried to run this tutorial script https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/flowers/sample.sh
Ran into this error:
***#***:~/git/cloudml-samples/flowers$ ./sample.sh
Your active configuration is: [cloudshell-4691]
Using job id: flowers_***_20170113_162148
python trainer/preprocess.py \
--input_dict "$DICT_FILE" \
--input_path "gs://cloud-ml-data/img/flower_photos/eval_set.csv" \
--output_path "${GCS_PATH}/preproc/eval" \
--cloud
WARNING:root:Using fallback coder for typehint: Any.
WARNING:root:Using fallback coder for typehint: Any.
WARNING:root:Using fallback coder for typehint: Any.
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting google-cloud-dataflow==0.4.4
Using cached google-cloud-dataflow-0.4.4.zip
Saved /tmp/tmpSoHiTi/google-cloud-dataflow-0.4.4.zip
Successfully downloaded google-cloud-dataflow
# Takes about 30 mins to preprocess everything. We serialize the two
Traceback (most recent call last):
File "trainer/preprocess.py", line 436, in <module>
main(sys.argv[1:])
File "trainer/preprocess.py", line 432, in main
run(arg_dict)
File "trainer/preprocess.py", line 353, in run
p.run()
File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 159, in run
return self.runner.run(self)
File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/apache_beam/runners/dataflow_runner.py", line 195, in run
% getattr(self, 'last_error_msg', None), self.result)
apache_beam.runners.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed:
(b85b0a598a3565cb): Workflow failed.
I was not able to get any clue where I was doing wrong from the Error Log of GoogleCloud Dataflow
Appreciate any answer and help for troubleshooting.
Enable the Dataflow API. In the Pantheon top search box typing "dataflow api" will take you to a window where you can click "Enable API".
I think this will fix it for you. I disabled my Dataflow API and got the same error as you, then when it was re-enabled the problem went back away.

ponydebugger error when installing

I got some error message after the command :
curl -sk https://cloud.github.com/downloads/square/PonyDebugger/bootstrap-ponyd.py | \
python - --ponyd-symlink=/usr/local/bin/ponyd ~/Library/PonyDebugger
this is my termial tell me,
Overwriting /Users/hokila/Library/PonyDebugger/lib/python2.7/orig-prefix.txt with new content
New python executable in /Users/hokila/Library/PonyDebugger/bin/python
Traceback (most recent call last):
File "<stdin>", line 2462, in <module>
File "<stdin>", line 944, in main
File "<stdin>", line 1045, in create_environment
File "<stdin>", line 1361, in install_python
File "<stdin>", line 435, in copyfile
File "<stdin>", line 412, in copyfileordir
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 128, in copy2
copyfile(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: '/Users/hokila/Library/PonyDebugger/.Python'
It seems the python version error,so I update my python to 2.7.3.
then I still got the same error.How could I solve this?
This problem also occured to me and I was almost desperate. I reinstalled Python 2.7.2 and it worked fine.
I tried to solve this problem manually before using the instructions in README_ponyd.rst and I was also partially successful:
Development installation ````````````````````````
If you already have PonyDebugger git repo checked out you can can set
up a virtualenv manually and have your ponyd installation point to
your existing checkout. For demonstration we assume $VENV is set
to your intended install path and $PONYDEBUGGER_PATH is set to
your PonyDebugger git checkout::
# if you don't already have virtualenv installed
sudo easy_install
virtualenv
virtualenv "$VENV"
source "$VENV/bin/activate"
pip install -e "$PONYDEBUGGER_PATH"
# to ensure your shell knows ponyd exists hash -r
To run this ponyd you can either activate your environment by source
"$VENV/bin/activate" and ponyd will be added to your path. You
can also just call it directly via $VENV/bin/ponyd without
activating first.
You should install virtualenv before. You can try combination of both steps. Hope it helps...

Building OpenCV 2.2 html documentation

I've followed the directions here to build html documentation https://code.ros.org/trac/opencv/browser/trunk/opencv/doc/README.txt (actually my local copy that came with OpenCV 2.2, but they are the same), but I get this error after running sh buildall:
parsing
Traceback (most recent call last):
File "latex.py", line 714, in <module>
fulldoc = latexparser(sys.argv[1])
File "/home/user/OpenCV-2.2.0/doc/latex2sphinx/latexparser.py", line 114, in latexparser
tokens = tokenize(filename)
File "/home/user/OpenCV-2.2.0/doc/latex2sphinx/latexparser.py", line 106, in tokenize
pickle.dump(r, open(cache_filename, 'w'))
IOError: [Errno 2] No such file or directory: 'parse-cache/c83e178b805f6b01ff8d55cda4bd4a29'
The pdf file built fine but I prefer html.
I'm using Ubuntu 10.10.
I was able to get this working by simply doing "mkdir parse-cache" in the opencv/doc/latex2sphinx directory.

Resources