I try to run the bigscience Bloom AI Model on my Macbook M1 Max 64GB, freshly installed pytorch for Mac M1 chips and Python 3.10.6 running.
I'm not able to get any output at all.
With other AI Models I have the same issue and I really don't know how I should fix it.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "mps" if torch.backends.mps.is_available() else "cpu"
if device == "cpu" and torch.cuda.is_available():
device = "cuda" #if the device is cpu and cuda is available, set the device to cuda
print(f"Using {device} device") #print the device
tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom")
model = AutoModelForCausalLM.from_pretrained("bigscience/bloom").to(device)
input_text = "translate English to German: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(device)
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
I've tried it with other Models (smaller bert Models) and also tried letting it just run on CPU without using the mps device at all.
Maybe anyone could help
It might be taking too long to get the output. Do you want to break it down to serial calls involving
a) embedding layer b) the 70 bloom blocks c) then the output layer norm and d) the token decoding?
An example to run this code is available at https://nbviewer.org/urls/arteagac.github.io/blog/bloom_local.ipynb .
It basically boils down to:
def forward(input_ids):
# 1. Create attention mask and position encodings
attention_mask = torch.ones(len(input_ids)).unsqueeze(0).bfloat16().to(device)
alibi = build_alibi_tensor(input_ids.shape[1], config.num_attention_heads,
torch.bfloat16).to(device)
# 2. Load and use word embeddings
embeddings, lnorm = load_embeddings()
hidden_states = lnorm(embeddings(input_ids))
del embeddings, lnorm
# 3. Load and use the BLOOM blocks sequentially
for block_num in range(70):
load_block(block, block_num)
hidden_states = block(hidden_states, attention_mask=attention_mask, alibi=alibi)[0]
print(".", end='')
hidden_states = final_lnorm(hidden_states)
#4. Load and use language model head
lm_head = load_causal_lm_head()
logits = lm_head(hidden_states)
# 5. Compute next token
return torch.argmax(logits[:, -1, :], dim=-1)
Please refer the linked notebook to get the implementation for functions used in the forward call.
I'm new to coding and using a raspberry pi. I've searched through many tutorials online and found a how to get OpenCV library into the pi itself and downloaded VSC on my laptop and the pi. The issue that I'm having is the code that I used on my laptop doesn't work the same on the pi. I've been getting errors on my code that dosen't show in in my laptop VSC.
the purpose is to display a live feed from the camera in the raspberry pi
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
ret, frame = cap.read()
frame = cv2.resize(frame, (0,0), fx=0.5,fy=0.5)
cv2.imshow("Frame",frame)
ch = cv2.waitKey(1)
if ch & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
(line 10) error: (-206:Bad flag (parameter or structure field)) Unrecognized or unsupported array type in function 'cvGetMat'
(line 9) error: (-215:Assertion failed) !ssize.empty() in function 'resize'
I think you have to use different device id in cv.VideoCapture(id)
because raspberry is using linux and at some point the id is not always 0
I can able to access the inbuilt laptop camera with the following code
import cv2
#import numpy as np
cap = cv2.VideoCapture(0)
while (True):
ret, frame = cap.read()
key = cv2.waitKey(20)
cv2.imshow("preview", frame)
if key == 27: # exit on ESC
break
cap.release()
cv2.destroyAllWindows()
I connected a USB camera. I gave 1 instead of 0 to access it. but it is showing an error. By changing it to between -1 and -99, I can access the inbuilt cam only. How can I access the USB cam? I checked the working condition of USB cam with cheese. It is working fine.
Previously I've used industrial cameras with Ethernet connections and distinct IP addresses for multiple camera setups. Now I'm attempting a multiple camera setup with OpenCV and I'm not sure how to match the OpenCV VideoCapture ID to a certain camera.
I should probably use my current situation as an example to make my question more clear. I currently have 3 cameras connected. I'm using Ubuntu 18.04 if that matters. Here is my output from lsusb (omitting everything except the 3 Logitech webcams I have connected):
$ lsusb
Bus 001 Device 013: ID 046d:0843 Logitech, Inc. Webcam C930e
Bus 001 Device 003: ID 046d:0843 Logitech, Inc. Webcam C930e
Bus 001 Device 006: ID 046d:0892 Logitech, Inc. OrbiCam
As you can see I have 2 C930es and one OrbiCam connected. Based on this very helpful post:
https://superuser.com/questions/902012/how-to-identify-usb-webcam-by-serial-number-from-the-linux-command-line
I found I could get the serial number of the cams like so:
$ sudo lsusb -v -d 046d:0843 | grep -i serial
iSerial 1 D2DF1D2E
iSerial 1 99A8F15E
$ sudo lsusb -v -d 046d:0892 | grep -i serial
iSerial 1 C83E952F
Great, so I now have a way to uniquely identify each camera based on the serial numbers stored in the cam's memory (D2DF1D2E, 99A8F15E, and C83E952F).
The problem is, opening a webcam connection in OpenCV is done as follows:
vidCapForCamX = cv2.VideoCapture(OPEN_CV_VID_CAP_ID_FOR_CAM_X)
vidCapForCamY = cv2.VideoCapture(OPEN_CV_VID_CAP_ID_FOR_CAM_Y)
vidCapForCamZ = cv2.VideoCapture(OPEN_CV_VID_CAP_ID_FOR_CAM_Z)
Where camera X, Y, and Z are the 3 cameras I need to use, each for a different determined purpose, and OPEN_CV_VID_CAP_ID_FOR_CAM_X, Y, and Z are the OpenCV VideoCapture IDs. Right now, I'm relating cameras to the OpenCV VideoCapture IDs with the following manual process:
1) Make a test script like this:
# cam_test.py
import numpy as np
import cv2
cap = cv2.VideoCapture(4)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
while True:
# Capture frame-by-frame
ret, frame = cap.read()
# Display the resulting frame
cv2.imshow('frame', frame)
keyPress = cv2.waitKey(10)
if keyPress == ord('q'):
break
# end if
# end while
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
2) Try numbers 0-99 for the VideoCapture parameter until I find the 3 magic numbers for my 3 attached cameras. In my current example they are 0, 2, and 4.
3) Each time I find a valid VideoCapture ID, wave my hand in front of each camera until I determine which one that VideoCapture ID is for, then write down which camera in my project that needs to correspond to, ex in my case:
0 => serial D2DF1D2E => cam X
2 => serial 99A8F15E => cam Y
4 => serial C83E952F => cam Z
4) Edit my code (or a stored config file or database field) so cam X uses VideoCapture ID 0, cam Y uses VideoCapture ID 2, etc.
I should clarify that cameras X, Y, and Z are in different positions and serve different purposes, i.e. if I use VideoCapture ID 4 for cam X the application wouldn't work (they have to be mapped a certain way as above).
Clearly for a production application this routine is not acceptable.
I realize I can do something like this:
import cv2
openCvVidCapIds = []
for i in range(100):
try:
cap = cv2.VideoCapture(i)
if cap is not None and cap.isOpened():
openCvVidCapIds.append(i)
# end if
except:
pass
# end try
# end for
print(str(openCvVidCapIds))
To get a list of the valid OpenCV VideoCapture IDs, but I still have to do the manual hand wave thing to determine which OpenCV VideoCapture IDs corresponds to each camera.
To make matters worse, swapping which camera is connected to which physical port on a device shuffles the OpenCV VideoCapture IDs, so if any camera connection is changed, or a cam is added or removed the manual process has to be repeated for all cameras.
So my question is, is there some genius way (in code, not a manual way) to relate the serial number of each camera or some other unique ID stored in the cam's memory to the magic numbers that OpenCV seems to come up with for VideoCapture IDs?
To put my question another way, I need to write a function camSerialNumToOpenCvVidCapId that could be used like so:
vidCapForCamX = cv2.VideoCapture(camSerialNumToOpenCvVidCapId(D2DF1D2E))
vidCapForCamY = cv2.VideoCapture(camSerialNumToOpenCvVidCapId(99A8F15E))
vidCapForCamZ = cv2.VideoCapture(camSerialNumToOpenCvVidCapId(C83E952F))
Is this possible and how could this be done?
P.S. I'm comfortable with OpenCV C++ or Python, any helpful answers using either would be greatly appreciated.
--- Edit ---
This question:
OpenCV VideoCapture device index / device number
Has a response (not accepted) that pertains to using Windows API calls, but I'm using Ubuntu.
--- Edit2 ---
# Micka, here is what I have for cameras in /dev/:
$ ls -l /dev/video*
crw-rw----+ 1 root video 81, 0 Nov 20 12:26 /dev/video0
crw-rw----+ 1 root video 81, 1 Nov 20 12:26 /dev/video1
crw-rw----+ 1 root video 81, 2 Nov 20 12:26 /dev/video2
crw-rw----+ 1 root video 81, 3 Nov 20 12:26 /dev/video3
crw-rw----+ 1 root video 81, 4 Nov 20 12:26 /dev/video4
crw-rw----+ 1 root video 81, 5 Nov 20 12:26 /dev/video5
I'm not sure if this helps
--- Edit3 ---
After considering this some more what I really need is a cam property in OpenCV to identify each camera uniquely. After getting a list of available VideoCapture IDs as mentioned above, if there was a property like:
serialNum = cv2.get(cv2.CAP_PROP_SERIAL_NUM)
Then it would be easy, but there does not seem to be such a property or anything similar (after checking PyCharm auto-complete for cv2.CAP_PROP_* and reading the OpenCV docs for VideoCapture).
For the solution you found, you need root privileges. On my setup with Ubuntu20 this is not required for:
udevadm info --name=/dev/video0
This outputs properties of first camera detected. Pipe it through "grep" to filter out specific property that is different for all cameras like "ID_SERIAL=". You can then use "cut" to remove beginning of this string "ID_SERIAL=" and leave just the value like:
udevadm info --name=/dev/video0 | grep ID_SERIAL= | cut -d "=" -f 2
In Python you can run external command to get this info like:
def get_cam_serial(cam_id):
# Prepare the external command to extract serial number.
p = subprocess.Popen('udevadm info --name=/dev/video{} | grep ID_SERIAL= | cut -d "=" -f 2'.format(cam_id),
stdout=subprocess.PIPE, shell=True)
# Run the command
(output, err) = p.communicate()
# Wait for it to finish
p.status = p.wait()
# Decode the output
response = output.decode('utf-8')
# The response ends with a new line so remove it
return response.replace('\n', '')
To acquire all the camera serial numbers, just loop through several camera ID's. On my setup trying camera ID 0 and 1 target the same camera. Also 2 and 4 target the second camera, so the loop can have 2 for step. Once all ID's are extracted, place them in a dictionary to be able to associate cam ID with serial number. The complete code could be:
serials = {}
FILTER = "ID_SERIAL="
def get_cam_serial(cam_id):
p = subprocess.Popen('udevadm info --name=/dev/video{} | grep {} | cut -d "=" -f 2'.format(cam_id, FILTER),
stdout=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
p.status = p.wait()
response = output.decode('utf-8')
return response.replace('\n', '')
for cam_id in range(0, 10, 2):
serial = get_cam_serial(cam_id)
if len(serial) > 6:
serials[cam_id] = serial
print('Serial numbers:', serials)
It is not very difficult to do. In Linux browse to the directory
/dev/v4l/by-id/
This directory lists all the webcams connected to your system with names like usb-046d_081b_31296650-video-index0 Copy this id and use it in your code in the following manner:
cv::VideoCapture camera;
camera.open("/dev/v4l/by-id/usb-046d_081b_31296650-video-index0");
cv::Mat frame;
camera >> frame;
For different cameras you can first note down their ids and then refer them in your code.
I purchased two webcams (Logitech C310 HD Webcam) to use with RaspberryPi (RPi 3 B+ model). When I run individual cameras, they are running fine but when I tried to run both cameras at the same time it didn't run. I came to know that it may be due to less power in Raspberry Pi, so I purchased a powered USB hub (Power USB hub). When I attached both cameras with raspberry pi through a power USB hub, it shows an error.
Unable to stop the stream: Invalid argument
OpenCV(3.4.1) Error: Assertion failed (size.width>0 && size.height>0) in imshow, file /home/pi/opencv-3.4.1/modules/highgui/src/window.cpp, line 356
Traceback (most recent call last):
File "two cameras simu.py", line 7, in <module>
cv2.imshow('frame1',frame1)
cv2.error: OpenCV(3.4.1) /home/pi/opencv-3.4.1/modules/highgui/src/window.cpp:356: error: (-215) size.width>0 && size.height>0 in function imshow
The code I used is:
import cv2
import numpy as np
cam1 = cv2.VideoCapture(1)
cam2 = cv2.VideoCapture(2)
while (1):
_,frame1 = cam1.read()
cv2.imshow('frame1',frame1)
_,frame2 = cam2.read()
cv2.imshow('frame2',frame2)
k = cv2.waitKey(5) & 0xFF
if k == 27:
break
cam1.release()
cam2.release()
cv2.destroyAllWindows()
While the same code I run in the laptop (in PyCharm) with the power USB hub attached, it works fine.
Why there is an error while trying to run two cameras with Raspberry Pi? How I can run two webcams using Raspberry Pi.
Try adding at the top
from imutils import VideoStream
import imutils
then change the input source from your cameras accordingly for example
cam1 = VideoStream(src=0).start()
hope this solves your problem