The problem is when I try to load a handmade dataset into Weka, I get the aforementioned error.the dataset style is as follows and it seems the style of the dataset has some problems.I've checked correct style of the dataset with "Iris" example but I don't know where the problem is.
Thank you in advance.
#RELATION link
#ATTRIBUTE personalinformation NUMERIC
#ATTRIBUTE educationalinformation NUMERIC
#ATTRIBUTE jobinformation NUMERIC
#ATTRIBUTE class {Strong,Weak}
#DATA
3,5,1,Strong
3,6,3,Strong
2,4,4,Strong
4,5,1,Strong
4,5,2,Strong
3,3,2,Strong
4,3,1,Strong
5,6,3,Strong
4,3,1,Strong
5,5,5,Strong
3,6,7,Strong
3,5,3,Strong
3,4,1,Strong
2,8,2,Strong
2,6,6,Strong
3,6,2,Strong
3,4,3,Strong
4,5,1,Strong
3,5,1,Strong
4,3,2,Strong
3,1,1,Weak
2,1,1,Weak
3,1,1,Weak
2,4,1,Weak
2,1,1,Weak
3,1,1,Weak
3,1,1,Weak
2,4,1,Weak
3,1,1,Weak
3,1,1,Weak
2,5,1,Weak
3,2,1,Weak
3,3,1,Weak
2,1,1,Weak
3,3,1,Weak
2,1,1,Weak
2,1,1,Weak
2,1,1,Weak
Related
I have this code. But can't understand why answer always empty with any input texts. Thank you very much for the help:
import torch
from transformers import AutoTokenizer, AutoModel
# Load the pre-trained BERT model
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")
# Encode the article text and the question
article_text = "The Sun is a star"
question = "What is The Sun?"
encoded_article_text = tokenizer.encode(article_text, return_tensors="pt")
encoded_question = tokenizer.encode(question, return_tensors="pt")
# Concatenate the article and the question
input_ids = torch.cat([encoded_article_text, encoded_question], dim=-1)
# Predict the start and end position of the answer in the article
output = model(input_ids)
start_logits = output[0][:, :encoded_article_text.size()[1]]
end_logits = output[1][:, :encoded_article_text.size()[1]]
# Find the answer by selecting the highest scoring start and end position
start_index = torch.argmax(start_logits)
end_index = torch.argmax(end_logits)
answer_tokens = encoded_article_text[0, start_index:end_index+1]
answer = tokenizer.decode(answer_tokens)
print("Answer:", answer)
tried change inputs with different texts and answers
Found a Solution, left it as an answer to this question down below :)
Info about the project: Classification task with 2 classes.
I am trying to get the output of the fully connected layer of my model for each image I put into the model during runtime. I plan to use them after the model is done training or testing all images to visualize with UMAP.
The model:
#Load resnet
def get_model():
model = torchvision.models.resnet50(pretrained=True)
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 2)
return model
The relevant part of pl module:
class classifierModel(pl.LightningModule):
def __init__(self, model):
super().__init__()
self.model = model
self.learning_rate = 0.0001
def training_step(self, batch, batch_idx):
x= batch['image']
y = batch['targets']
x_hat = self.model(x)
output = nn.CrossEntropyLoss()
loss= output(x_hat,y)
return loss
def test_step(self, batch, batch_idx):
x= batch['image']
y = batch['targets']
x_hat = self.model(x)
Is it possible to do this by adding a empty list to the init of the pl module and then add the output after x_hat = model(x) is executed?
How would i know if after x_hat = model(x) is executed, the out_features aren't immediatly deleted/discarded ?
x_hat is this vector and is [batch_size, 2048]. So just modify your training step to also return x_hat.
class classifierModel(pl.LightningModule):
def __init__(self, model):
super().__init__()
self.model = model
self.learning_rate = 0.0001
self.fc_outputs = []
def training_step(self, batch, batch_idx):
x= batch['image']
y = batch['targets']
x_hat = self.model(x)
self.fc_outputs.append(x_hat)
output = nn.CrossEntropyLoss()
loss= output(x_hat,y)
return loss
The values of x_hat will not be deleted unless you explicitly call del x_hat BEFORE assigning these values elsewhere. In the case where you have already assigned the values of x_hat to another variable (in your case it sounds like you want to append it to a list) the memory addresses associated with the values are not deallocated because there is still a variable that references these addresses even after the original variable referencing them (x_hat may have been deleted). In this way, python is relatively safe in terms of memory referencing because it dynamically computes when memory addresses / values are no longer needed at runtime.
I was able to do this using a forward hook on the avgpool layer and saving the output on each test_step like described here :
#Define Hook:
def get_features(name):
def hook(model, input, output):
features[name] = output.detach()
return hook
Now when I load my model, I register the hook:
#Load resnet model:
def get_model():
model = models.resnet50(pretrained=True)
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 2)
model.avgpool.register_forward_hook(get_features('feats')) #register the hook
return model
I did not need to change the init of the pytorch lightning model but the test step function:
FEATS = []
# placeholder for batch features
features = {}
class classifierModel(pl.LightningModule):
def __init__(self, model):
super().__init__()
self.model = model
self.learning_rate = 0.0001
def test_step(self, batch,batch_idx):
x= batch['image']
y = batch['targets']
x_hat = self.model(x)
FEATS.append(features['feats'].cpu().numpy()) #added this line to save output
now we have the output FEATS[0].shape --> (16, 2048, 1, 1) which is what i wanted to get(16 is the batchsize is use).
I would like to create a custom scanner for i18n-tasks that can detect enums declared as hashes in models.
My enum declaration pattern will always be like this:
class Conversation < ActiveRecord::Base
enum status: { active: 0, archived: 1}, _prefix: true
enum subject: { science: 0, literature: 1, music: 2, art: 3 }, _prefix: true
end
The enums will always be declared as hashes, and will always have a numerical hash value, and will always have the option _prefix: true at the end of the declaration. There can be any number of values in the hash.
My custom scanner currently looks like this:
require 'i18n/tasks/scanners/file_scanner'
class ScanModelEnums < I18n::Tasks::Scanners::FileScanner
include I18n::Tasks::Scanners::OccurrenceFromPosition
# #return [Array<[absolute key, Results::Occurrence]>]
def scan_file(path)
text = read_file(path)
text.scan(/enum\s([a-zA-Z]*?):\s\{.*\W(\w+):.*\}, _prefix: true$/).map do |prefix, attribute|
occurrence = occurrence_from_position(
path, text, Regexp.last_match.offset(0).first)
model = File.basename(path, ".rb") #.split('/').last
name = prefix + "_" + attribute
["activerecord.attributes.%s.%s" % [model, name], occurrence]
end
end
end
I18n::Tasks.add_scanner 'ScanModelEnums'
However this is only returning the very last element of each hash:
activerecord.attributes.conversation.status_archived
activerecord.attributes.conversation.subject_art
How can I return all the elements of each hash? I am wanting to see a result like this:
activerecord.attributes.conversation.status_active
activerecord.attributes.conversation.status_archived
activerecord.attributes.conversation.subject_science
activerecord.attributes.conversation.subject_literature
activerecord.attributes.conversation.subject_music
activerecord.attributes.conversation.subject_art
For reference, the i18n-tasks github repo offers an example of a custom scanner.
The file scanner class that it uses can be found here.
This works:
def scan_file(path)
result = []
text = read_file(path)
text.scan(/enum\s([a-zA-Z]*?):\s\{(.*)}, _prefix: true$/).each do |prefix, body|
occurrence = occurrence_from_position(path, text,
Regexp.last_match.offset(0).first)
body.scan(/(\w+):/).flatten.each do |attr|
model = File.basename(path, ".rb")
name = "#{prefix}_#{attr}"
result << ["activerecord.attributes.#{model}.#{name}", occurrence]
end
end
result
end
It's similar to your 'answer' approach, but uses the regex to get all the contents between '{...}', and then uses another regex to grab each enum key name.
The probable reason your 'answer' version raises an error is that it is actually returning a three-dimensional array, not two:
The outer .map is an array of all iterations.
Each iteration returns retval, which is an array.
Each element of retail is an array of ['key', occurrence] pairs.
This isn't the answer, this is just the other attempt I made, which outputs a two dimensional array instead of a single array:
require 'i18n/tasks/scanners/file_scanner'
class ScanModelEnums < I18n::Tasks::Scanners::FileScanner
include I18n::Tasks::Scanners::OccurrenceFromPosition
# #return [Array<[absolute key, Results::Occurrence]>]
def scan_file(path)
text = read_file(path)
text.scan(/enum\s([a-zA-Z]*?):\s\{(.*)\}, _prefix: true/).map do |prefix, attributes|
retval = []
model = File.basename(path, ".rb")
names = attributes.split(",").map!{ |e| e.strip; e.split(":").first.strip }
names.each do |attribute|
pos = (Regexp.last_match.offset(0).first + 8 + prefix.length + attributes.index(attribute))
occurrence = occurrence_from_position(
path, text, pos)
name = prefix + "_" + attribute
# p "================"
# p type
# p message
# p ["activerecord.attributes.%s.%s" % [model, name], occurrence]
# p "================"
retval.push(["activerecord.attributes.%s.%s" % [model, name], occurrence])
end
retval
end
end
end
I18n::Tasks.add_scanner 'ScanModelEnums'
This however gives me an error for the second detected attribute:
gems/i18n-tasks-0.9.34/lib/i18n/tasks/scanners/results/key_occurrences.rb:48:in `each': undefined method `path' for ["activerecord.attributes.conversation.status_archived", Occurrence(app/models/project.rb:3:32:98::)]:Array (NoMethodError)
I'm trying to integrate django-allauth with a custom user model (subclassed AbstractUser, but when I test the signup form I get an integrity error due to field (date_of_birth) being null, but the value submitted was u'1976-4-6'
I'm learning the new custom user stuff, as well as class-based views as I'm learning django-allauth, so I'm confident that I'm doing something wrong, but after a couple days of reading the github issues, the few tutorials, readthedocs, and stackoverflow questions I still have no clear idea of what I'm doing wrong (well I know one thing I'm doing wrong: trying different solutions here and there, so I definitely have a miss-mosh of implementations)
But, I can't find a good answer on how to integrate allauth with a subclassed AbstractUser, so if anyone could enlighten me, I would really appreciate it.
(Note - the site is more or less working when I log in as a user that I've loaded via fixtures, so please assume that non-django-allauth omissions are omissions - if you need clarification on something not below, I will happily edit)
settings.py
AUTH_USER_MODEL = 'userdata.CtrackUser'
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_SIGNUP_FORM_CLASS = 'userdata.forms.SignupForm'
LOGIN_REDIRECT_URL = '/profile'
SOCIALACCOUNT_QUERY_EMAIL = True
SOCIALACCOUNT_AUTO_SIGNUP = False
ACCOUNT_USER_MODEL_USERNAME_FIELD = 'username'
userdata/models.py
class CtrackUser(AbstractUser):
date_of_birth = models.DateField(help_text='YYYY-MM-DD format')
gender = models.CharField(max_length=2,
choices=settings.GENDER_CHOICES, blank=True)
race = models.CharField(max_length=2, choices=settings.RACE_CHOICES, null=True, blank=True)
condition = models.ForeignKey(Condition, null=True, blank=True)
location = models.CharField(max_length=255, null=True, blank=True)
my_symptoms = models.ManyToManyField(Symptom)
is_admin = models.BooleanField(default=False)
created = models.DateTimeField(auto_now_add=True)
modified = models.DateTimeField(auto_now=True)
userdata/forms.py
from django import forms
from django.conf import settings
from django.contrib.auth import get_user_model
from allauth.account.forms import SetPasswordField, PasswordField
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Submit
from medical.models import Condition
class SignupForm(forms.Form):
email = forms.EmailField(required=True,)
username = forms.CharField(max_length=80,required=True,)
password1 = SetPasswordField()
password2 = PasswordField()
first_name = forms.CharField(max_length=100,required=False,)
last_name = forms.CharField(max_length=100, required=False,)
date_of_birth = forms.DateField()
gender = forms.TypedChoiceField(
choices=settings.GENDER_CHOICES,
widget=forms.Select(attrs={'class': 'input-lg'}),
required=False,)
race = forms.TypedChoiceField(
choices=settings.RACE_CHOICES,
widget=forms.Select(attrs={'class': 'input-lg'}),
required=False,)
location = forms.CharField(max_length=255,required=False,)
condition = forms.ModelChoiceField(
queryset=Condition.objects.all(),
widget=forms.Select(attrs={'class': 'input-lg'}),
empty_label='Select condition (optional)'
)
class Meta:
model = get_user_model() # use this function for swapping user model
fields = ('email', 'username', 'password1', 'password2', 'first_name', 'last_name',
'date_of_birth', 'gender', 'race', 'location', 'condition', 'confirmation_key',)
def __init__(self, *args, **kwargs):
super(SignupForm, self).__init__(*args, **kwargs)
self.helper = FormHelper()
self.helper.form_id = 'signup_form'
self.helper.label_class = 'col-xs-6'
self.helper.field_class = 'col-xs-12'
self.helper.form_method = 'post'
self.helper.form_action = 'accounts_signup'
self.helper.add_input(Submit('submit', 'Sign up'))
def signup(self, request, user, model):
user.username = self.cleaned_data['username']
user.first_name = self.cleaned_data['first_name']
user.last_name = self.cleaned_data['last_name']
model.date_of_birth = self.cleaned_data['date_of_birth']
model.gender = self.cleaned_data['gender']
model.race = self.cleaned_data['race']
model.location = self.cleaned_data['location']
model.condition = self.cleaned_data['condition']
model.save()
user.save()
templates/allauth/account/signup.html
<form id="signup_form" method="post" action="{% url 'account_signup' %}" class="form-inline">
{% csrf_token %}
{% crispy form %}
{% if redirect_field_value %}
<input type="hidden" name="{{ redirect_field_name }}" value="{{ redirect_field_value }}" />
{% endif %}
{# <div class="form-actions">#}
{# <button class="btn btn-primary" type="submit">Sign Up</button>#}
{# </div>#}
</form>
POST data
u'condition' [u'1']
u'confirmation_key' [u'']
u'date_of_birth' [u'1976-4-6']
u'email' [u'1#bt.co']
u'first_name' [u'One']
u'gender' [u'']
u'last_name' [u'Person']
u'location' [u''] u'password1' [u'123456']
u'password2' [u'123456']
u'race' [u'']
u'submit' [u'Sign up']
u'username' [u'gn']
Error generated (note difference from post data)
Exception Type: IntegrityError at /accounts/signup/
Exception Value: null value in column "date_of_birth" violates not-null constraint
DETAIL: Failing row contains (19, pbkdf2_sha256$12000$exNVzh4QI0Rb$mCTz9Tc+TIBbD8+lIZs2B3hqjxd+qmI..., 2014-07-02 16:27:43.751428+00, f, gn, One, Person, 1#bt.co, f, t, 2014-07-02 16:27:43.751473+00, null, , null, null, null, f, 2014-07-02 16:27:43.833267+00, 2014-07-02 16:27:43.83329+00).
Full traceback here:
https://gist.githubusercontent.com/hanleybrand/ee260b53dfb404f5055a/raw/3325dc746120c4f7521b9b976abce45dd7d71a77/gistfile1.txt
The answer -- which I'm still figuring out -- seems to be that if you are saving a model that contains field types that allauth.account.adapter.DefaultAccountAdapter doesn't handle correctly (e.g. any field that lacks a __getitem__ attribute, like models.DateField) it is necessary to implement a custom adapter somewhat like below.
note: your subclassed abstract user model is the user that's passed in, so the best practice is to use the form data directly like user.email = data.get('email') rather than using the allauth internal methods used in the DefaultAccountAdapter class
userdata/adapter.py
class AccountAdapter(DefaultAccountAdapter):
def save_user(self, request, user, form, commit=False):
data = form.cleaned_data
user.email = data.get('email')
user.username = data.get('username')
# all your custom fields
user.date_of_birth = data.get('date_of_birth')
user.gender = data.get('gender')
if 'password1' in data:
user.set_password(data["password1"])
else:
user.set_unusable_password()
self.populate_username(request, user)
if commit:
user.save()
return user
I have a table which contains a column named DOB whose data type is DATETIME.The problem is that created_at is also a datetime but its data received through JSON is in this format
"created_at":"2013-02-02 11:57:42",
"dob":"2013-02-18T18:30:00Z"
Both the dates are in different format but both of them has datetime data type.
Now I'm using Datejs
to format the date which will not parse DOB format it can only parse created_at format.
What should I do now ?
Here's what I'm doing to parse through datejs
if(value.dob != null){
alert(value.dob);
d1 = Date.parse(value.dob);
alert(d1);
dob = d1.toString("M-d-yyyy");
}
And I'm getting this error on console:
: d1 is null
[Break On This Error]
dob = d1.toString("M-d-yyyy");
My controller:
def get_oi_report_contact
#contactlist = CaseOiReportMap.select("rc.*").where("case_oi_report_maps.oireport_identifier = ?",identifier)
.joins("LEFT JOIN case_oi_report_contacts_maps cm on cm.case_oi_report_map_id = case_oi_report_maps.id")
.joins("LEFT JOIN oi_report_contacts rc on rc.id = cm.oi_report_contact_id ")
respond_to do |format|
format.json { render :json => #contactlist.to_json }
end
end
I have also used monkey patch:
class ActiveSupport::TimeWithZone
def as_json(options = {})
strftime('%Y-%m-%d %H:%M:%S')
end
end
But it doesn't seem to work.
Use strftime to format the time string http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime