I have a column in a DataFrame with dates in yyyymmdd format and I need to change it permanently to yyyy-mm-dd.
How can I do that?
Given the info you provided in your comment, the column values can't be in the form yyyy-mm-dd since the column dtype is int64.
You can change the column dtype to be str, but the data won't be useful (ie you won't be able to do any date-calculations on it, though <, > should still work but lexicographically). If that's still what you wish and assuming df is the dataframe and the date column name is date:
def format_date_col(x):
x = str(x)
return '-'.join([x[:4], x[4:6], x[6:]])
# or maybe like that for better readability:
x = str(x)
return '{year}-{month}-{day}'.format(year=x[:4], month=x[4:6], day=x[6:])
df['date'] = df['date'].apply(format_date_col)
A better approach would be to use actual date dtype:
from datetime import datetime
def format_date_col(x):
return datetime.strptime(str(x), '%Y%m%d')
df['date'] = df['date'].apply(format_date_col)
print df['date'].dtype
>> datetime64[ns]
Related
I have a G1 table (Gramex) in UI which has columns with date and time but as string.
When the user sorts the respective column with the help of inbuilt sort option, it sorts the data as string not as date.
Approach 1
The easiest way would be if the date and time were formatted like YYYY-MM-DD HH:MM:SS. For example, '2022-09-04 03:02:01'. This is a string that can be sorted consistently with the datetime.
You can create such strings in Python with the datetime.isoformat() function.
Approach 2
If your dates are stored in a different format, e.g. 04-09-2022, etc., then IF the data is loaded from a file, not a database, then you can modify the data with the function: transform to convert the column to the right data format.
For example:
url:
continent:
pattern: /data
handler: FormHandler
kwargs:
url: data.csv
function: data.assign(date_col=pd.to_datetime(data[date_col]).strftime('%Y-%m-%dT%H:%M:%S'))
This converts the date_col (replace this with your date column name) from any date format Pandas recognizes into a YYYY-MM-DD HH:MM:SS type format that is sortable.
If the data is loaded from a database, then you DO need to convert it into a sortable format first.
Row 1: cell A is a concat of the date in B and the time in C. I generate these with CTRL+: and CTRL+SHIFT+: respectively. Google sheets does not treat this like a timestamp on the x axis of charts
Row 2: I discovered CTRL+ALT+SHIFT+: to do a full timestamp, now it has a real timestamp
The issue is, I have many rows of recorded data of the type in Row 1 -- is there any way to convert this into a 'time' format that Google Sheets will respect on the x-axis of charts? Using VALUE() just gives the date portion of the timestamp.
Kind of crazy how much trouble this is causing me, is there really no date_parse(string_format) type function I can call?
EDIT:
this is ridiculous, just going to export and use python
instead VALUE use TIMEVALUE and then format it internally to time
or:
=TEXT(TIMEVALUE(A1); "hh:mm:ss")
for arrayformula:
=ARRAYFORMULA(IF(A1:A="",,TEXT(TIMEVALUE(A1:A); "hh:mm:ss")))
for timestamp > date use DATEVALUE
Need to print date in xlsx file mm/dd/yyyy format, added the format using add_format method, but its printing as '2019-09-27 elabrated date
workbook = ::WriteXLSX.new(file, strings_to_urls: false)
worksheet = workbook.add_worksheet('Sheet 1')
row=column=1
date_val = Date.today
date_format = workbook.add_format({'num_format': 'dd/mm/yy'})
worksheet.write(1 + row, column, date_val, date_format)
workbook.close
then i tried write_date_time method, code sample below
workbook = ::WriteXLSX.new(file, strings_to_urls: false)
worksheet = workbook.add_worksheet('Sheet 1')
row=column=1
date_val = Date.today
date_format = workbook.add_format({'num_format': 'dd/mm/yy'})
worksheet.write_date_time(1 + row, column, item[:value]&.iso8601, date_format)
workbook.close
The issue with what you are doing is that you are passing a date object, but if you have read the Date Time documentation then the first 2 lines says this -
A date/time in Excel is a real number plus an Excel number format.
WriteXLSX doesn’t automatically convert date/time strings in write() to an Excel date/time.
Now, to answer your question you need to convert the date to a number which the Excel understands.
date_time = DateTime.now.strftime('%Y-%m-%dT00:00:00.000Z')
I have added 00:00:00.000 as you are not concerned with time if you want time also then check strftime documentation
date_time_number = worksheet.convert_date_time(date_time)
format2 = workbook.add_format(:num_format => 'dd/mm/yy')
worksheet.write(row, col, number, format2)
This should solve it for you
This helped me understand how to use convert_date_time and its mention is at the bottom of the Date Time documentation
I am using the below query with date filtering, but I am getting wrong result.
SELECT * FROM TRANSACTIONSHISTORY
WHERE DATE > "29-01-2015 12:00:00"
AND DATE < "30-01-2015 00:00:00" AND USERID=abc
I am getting result with date column with value of 29-Jan-2016 records, what am I missing here, can any one help me to get out of this value.
The date format in your SQL will not work because SQLite doesn't have a native datetime type, so it's generally stored either as a string, in YYYY-MM-DD HH:MM:SS.SSS format, or as an numeric value representing the number of seconds since 1970-01-01 00:00:00 UTC. See date and time types on SQLite.org. Note that if you're using the string representation that the sequence is year, month, day (which, when sorting/querying this string field, the this alphanumeric string will sort correctly by year first, then month, and then day, which is critical when doing queries like yours).
If you really stored dates in the database as a string in the DD-MM-YYYY HH:MM:SS format, you should consider changing the format in which you saved the values into one of the approved date formats. It will make the date interactions with the database much, much easier, allowing queries like the one you asked for (though, obviously, with DD-MM-YYYY replaced with YYYY-MM-DD format).
You have cast your string to Date
SELECT * FROM TRANSACTIONSHISTORY WHERE DATE between Datetime('29-01-2015 12:00:00') and Datetime('30-01-2015 00:00:00') AND USERID=abc
The first answer is exactly what you need. What you did in your code would be comparing strings using ASCII values.
I would recommend you to use the linux time stamps like: 1453818208, which is easier to save and compare. In addition, it can always be translated to human-readable dates like: 29-01-2015 12:00:00.
SELECT * FROM TRANSACTIONSHISTORY
WHERE DATE > "29-01-2015 12:00:00"
AND DATE < "30-01-2015 00:00:00" AND USERID=abc
I hope this helps you :)
Try this first try without Time,after that try date and time both , Hope i will work for you
SELECT TRANSACTIONSHISTORY
FROM SHIPMENT
WHERE DATE
BETWEEN '11-15-2010'
AND '30-01-2015'
// you can try this one also
SELECT * FROM TRANSACTIONSHISTORY WHERE DATE BETWEEN "2011-01-11" AND "2011-8-11"
I'm currently given three values in a table
A date value in the format of %dd-%mname-%yy (i.e 06-may-05), and am parsing that using Date.parse(input,true) to fix the issue with the leading values.
I'm then given a time value in the form of %hh:%mm:%ss.%ms (the ms of which I can take or leave) and a third value of a GMT offset.
I can't really see anyway to convert these three values into a single DateTime object that would allow me to manipulate it using the range of ruby tools without first parsing the second value to time, somehow changing the offset ((given as a + or - n value) as in +2 or -6)to a signed int and then applying it and then parsing this all to a super dateTime object.
There's got to be a better way. Is there?
Chronic may be able to parse this (if you concatenate everything in one string, maybe with some modifications) but I haven't checked.
Okay in order to create a dateTime value with the time and the date given and to take into account an offset you need the following code
d = DateTime.parse(dateVal+" "+TimeVal)
offset = Rational(offset_val,24)
d = d.new_offset(offset)
So take your date, given to you as say 05 May 2010 and a timeval in the form hh:mm:ss
With an offset of +- any value, for this instance say -8
Then this code will generate you a new date object, offset to the amount you require