Expanding a list contained in a column, so that each element of the list corresponds to its own column and is represented as a binary variable - binary-data

Question

I have a dataframe that looks like this:
skill_list name profile 561 904 468 875 737 402 882...
[561, 564, 632, 859] Aaron Weidele wordpress developer 0 0 0 0 0 0 0
[737, 399, 882, 1086, 5...]Abdelrady Tantawy full stack developer 0 0 0 0 0 0 0
[904, 468, 783, 1120, 8...]Abhijeet A Mulgund machine learning dev... 0 0 0 0 0 0 0 [468] Abhijeet Tiwari salesforce programmi... 0 0 0 0 0 0 0
[518, 466, 875, 445, 402..]Abhimanyu Veer A...machine learning devel...0 0 0 0 0 0 0
The skill_list column contains a list of encoded skills, which correspond to a developer. I would like to expand each list contained within the skill_list column, so that each encoded skill is represented within its own column as a binary variable (1 for on and 0 for off). Expected output would be:
skill_list name profile 561 904 468 875 737 402 882...
[561, 564, 632, 859] Aaron Weidele wordpress developer 1 0 0 0 0 0 0
[737, 399, 882, 1086, 5...]Abdelrady Tantawy full stack developer 0 0 0 0 1 0 1
[904, 468, 783, 1120, 8...]Abhijeet A Mulgund machine learning dev... 0 1 1 0 0 0 0 [468] Abhijeet Tiwari salesforce programmi... 0 0 1 0 0 0 0
[518, 466, 875, 445, 402..]Abhimanyu Veer A...machine learning devel...0 0 0 0 0 1 0
I've tried:
for index, row in df_vector_matrix["skill_list"].items():
for item in row:
for col in df_vector_matrix.columns:
if item == col:
df_vector_matrix.loc[item, col] = "1"
else:
0
I would really appreciate the help!

Answer 1

You can try MultiLabelBinarizer from sklearn.
Below example might help.
from sklearn.preprocessing import MultiLabelBinarizer
lb = MultiLabelBinarizer()
lb_res = lb.fit_transform(df_vector_matrix['skill_list'])
# convert result into dataframe
res = pd.DataFrame(lb_res,columns=lb.classes_)
# concatenate data result and original dataframe
df_vector_matrix = pd.concat([df_vector_matrix,res],axis=1)
Below is with the example dataframe, where the col column have list values.
>>> import pandas as pd
>>> from sklearn.preprocessing import MultiLabelBinarizer
>>> d ={'col':[[1,2,3],[2,3,4,5],[2]],'name':['abc','vdf','rt']}
>>> df = pd.DataFrame(d)
>>> df
col name
0 [1, 2, 3] abc
1 [2, 3, 4, 5] vdf
2 [2] rt
>>> lb = MultiLabelBinarizer()
>>> lb_res = lb.fit_transform(df['col'])
>>> res = pd.DataFrame(lb_res,columns=lb.classes_)
>>> pd.concat([df,res],axis=1)
col name 1 2 3 4 5
0 [1, 2, 3] abc 1 1 1 0 0
1 [2, 3, 4, 5] vdf 0 1 1 1 1
2 [2] rt 0 1 0 0 0
>>>