Now I need to find a particular entry in a journal using a CL program. The way I use to locate it is to DSPJRNE to put the journal entries in an output file, then use OPNQRYF to filter the desired one. The file is uniquely keyed so my plan is to compare the journal entry data with the key. The problem is that one of the key is a packed decimal so in the journal entry it is treated as hexadecimal code of characters and displayed as some strange symbols. So in order to compare the strings I need to convert the packed decimal key into the corresponding characters. How to achieve this in CL? If using CL is not possible, what about RPG?
To answer your immediate question, the CVTCH MI instruction will convert hex to char but I would not go that route; neither in CL nor RPG. Rather, I would take James' advice with a few additional steps.
DSPJRNE OUTFILE(QTEMP/DSPJRNE)
QRY input file DSPJRNE, output file QRYJRNE, select only JOESD
CRTDUPOBJ PRODUCTION_FILE QTEMP/JRNF DATA(*NO)
CPYF QRYJRNE JRNF FMTOPT(*NOCHK)
This will give you an externally described file with the exact same layout as your production file. You can query that, etc.
If you are pulling journal entries for a specific file you can dump them into an externally described file with a clever use of SQL:
CREATE TABLE QTEMP/QADSPJRN LIKE QSYS/QADSPJRN
ALTER TABLE QTEMP/QADSPJRN DROP COLUMN JOESD
CREATE TABLE QTEMP/DSPJRNE AS (SELECT * FROM QTEMP/QADSPJRN, FILE-LIB/FILE)
WITH NO DATA
DSPJRNE ... OUTPUT(*OUTFILE) OUTFILFMT(*TYPE1) OUTFILE(QTEMP/DSPJRNE)
ENDDTALEN(*CALC)
Related
I have a geometry column of a geodataframe populated with polygons and I need to upload these to Snowflake.
I have been exporting the geometry column of the geodataframe to file and have tried both CSV and GeoJSON formats, but so far I either always get an error the staging table always winds up empty.
Here's my code:
design_gdf['geometry'].to_csv('polygons.csv', index=False, header=False, sep='|', compression=None)
import sqlalchemy
from sqlalchemy import create_engine
from snowflake.sqlalchemy import URL
engine = create_engine(
URL(<Snowflake Credentials Here>)
)
with engine.connect() as con:
con.execute("PUT file://<path to polygons.csv> #~ AUTO_COMPRESS=FALSE")
Then on Snowflake I run
create or replace table DB.SCHEMA.DESIGN_POLYGONS_STAGING (geometry GEOGRAPHY);
copy into DB.SCHEMA."DESIGN_POLYGONS_STAGING"
from #~/polygons.csv
FILE_FORMAT = (TYPE = CSV FIELD_DELIMITER = '|' SKIP_HEADER = 1 compression = None encoding = 'iso-8859-1');
Generates the following error:
"Number of columns in file (6) does not match that of the corresponding table (1), use file format option error_on_column_count_mismatch=false to ignore this error File '#~/polygons.csv.gz', line 3, character 1 Row 1 starts at line 2, column "DESIGN_POLYGONS_STAGING"[6] If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client."
Can anyone identify what I'm doing wrong?
Inspired by #Simeon_Pilgrim's comment I went back to Snowflake's documentation. There I found an example of converting a string literal to a GEOGRAPHY.
https://docs.snowflake.com/en/sql-reference/functions/to_geography.html#examples
select to_geography('POINT(-122.35 37.55)');
My polygons looked like strings describing Polygons more than actual GEOGRAPHYs so I decided I needed to be treating them as strings and then calling TO_GEOGRAPHY() on them.
I quickly discovered that they needed to be explicitly enclosed in single quotes and copied into a VARCHAR column in the staging table. This was accomplished by modifying the CSV export code:
import csv
design_gdf['geometry'].to_csv(<path to polygons.csv>,
index=False, header=False, sep='|', compression=None, quoting=csv.QUOTE_ALL, quotechar="'")
The staging table now looks like:
create or replace table DB.SCHEMA."DESIGN_POLYGONS_STAGING" (geometry VARCHAR);
I ran into further problems copying into the staging table related to the presence of a polygons.csv.gz file I must have uploaded in a previous experiment. I deleted this file using:
remove #~/polygons.csv.gz
Finally, converting the staging table to GEOGRAPHY
create or replace table DB.SCHEMA."DESIGN_GEOGRAPHY_STAGING" (geometry GEOGRAPHY);
insert into DB.SCHEMA."DESIGN_GEOGRAPHY"
select to_geography(geometry)
from DB.SCHEMA."DESIGN_POLYGONS_STAGING"
and I wound up with a DESIGN_GEOGRAPHY table with a single column of GEOGRAPHYs in it. Success!!!
I am new to Neo4j and graph database. While trying to import a few relationships from a CSV file, I can see that there are no records, even when the file is filled with enough data.
LOAD CSV with headers FROM 'file:/graphdata.csv' as row WITH row
WHERE row.pName is NOT NULL
MERGE(transId:TransactionId)
MERGE(refId:RefNo)
MERGE(kewd:Keyword)
MERGE(accNo:AccountNumber {bName:row.Bank_Name, pAmt:row.Amount, pName:row.Name})
Followed by:
LOAD CSV with headers FROM 'file/graphdata.csv' as row WITH row
WHERE row.pName is NOT NULL
MATCH(transId:TransactionId)
MATCH(refId:RefNo)
MATCH(kewd:Keyword)
MATCH(accNo:AccountNumber {bName:row.Bank_Name, pAmt:row.Amount, pName:row.Name})
MERGE(transId)-[:REFERENCE]->(refId)-[:USED_FOR]->(kewd)-[:AGAINST]->(accNo)
RETURN *
Edit (table replica):
TransactionId Bank_Name RefNo Keyword Amount AccountNumber AccountName
12345 ABC 78 X 1000 5421 WE
23456 DEF X 2000 5471
34567 ABC 32 Y 3000 4759 HE
Is it likely the case that the Nodes and relationships are not created at all? How do I get all these desired relationships?
Neither file:/graphdata.csv nor file/graphdata.csv are legal URLs. You should use file:///graphdata.csv instead.
By default, LOAD CSV expects a "csv" file to consist of comma separated values. You are instead using a variable number of spaces as a separator (and sometimes as a trailer). You need to either:
use a single space as the separator (and specify an appropriate FIELDTERMINATOR option). But this is not a good idea for your data, since some bank names will likely also contain spaces.
use a comma separator (or some other character that will not occur in your data).
For example, this file format would work better:
TransactionId,Bank_Name,RefNo,Keyword,Amount,AccountNumber,AccountName
12345,ABC,78,X,1000,5421,WE
23456,DEF,,X,2000,5471
34567,ABC,32,Y,3000,4759,HE
Your Cypher query is attempting to use row properties that do not exist (since the file has no corresponding column headers). For example, your file has no pName or Name headers.
Your usage of the MERGE clause is probably not doing what you want, generally. You should carefully read the documentation, and this answer may also be helpful.
I have a sequential file having record length of 11. I have a field in-between starting from 9th position till 11th position and is of PIC 9(03). I want to delete all the records where I have same data in above specified location. This needs to be done using JCL only. Any utility can be used but should be supporting in microfocus cobol. See the example below:
Example File:
Rob ,d,012
Mike ,h,013
Kim ,g,014
Bob ,k,014
Wiz ,t,015
In the above example I want to delete rows for Kim and Mike as it is having same value for the location i.e. 014 and final output should be:
Rob ,d,012
Mike ,h,013
Wiz ,t,015
Try these statements in the SYSIN DD of DFSORT utility.
SORT FIELDS=COPY
OMIT COND=(9,3,ZD,EQ,014)
Microfocus use the mfsort utility which emulates all the major functions of IBM's DFSORT product.
mfsort option copy
use input-file [record definition]
[org organization] [key structure]
give output-file [record definition]
[org organization] [key structure]
omit cond (9,3,nu,eq,014)
More details about mfsort can be found here.
checkout nsort from ordinal. also syncsort now precisely. these are cots utilities that can drop duplicate records.
Sort by the name, and add up the count of records by name.
Use the OUTFIL command to delete the records that had more than one record:
RECORD TYPE=F,LENGTH=11
INREC FIELDS=(1,11,C'0001') * Include a dummy field to sum up
SORT FIELDS=(9,3,BI,A) * Sort by the "id"
SUM FIELDS=(12,4,ZD) * Count the number of records by "id"
OUTFIL FILES=OUT,BUILD=(1,11),OMIT=(12,4,NE,C'0001') * Get rid of duplicated records
END
This should work on DFSORT or SYNCSORT, not sure about MFSORT.
My pipeline looks similar to the following:
parDo return list per processed line | beam.io.WriteToText
beam.io.WriteToText adds a new line after each list element. How can I remove this new line and have the values separated by comma so I will be able to build CSV file
Any help is very appreciated!
Thanks,
eilalan
To remove the newline char, you can use this:
beam.io.WriteToText(append_trailing_newlines=False)
But for adding commas between your values, there's no out-of-the-box feature on TextIO to convert to CSV. But, you can check this answer for a user defined PTransform that can be applied to your PCollection in order to convert dictionary data into csv data.
I want an alternative to running frequency for string variables because I also want to get a case number for each of the string value (I have a separate variable for case ID).
After reviewing the string values I will need to find them to recode which is the reason I need to know the case number.
I know that PRINT command should do what I want but I get an error - is there any alternative?
PRINT / id var2 .
EXECUTE.
>Error # 4743. Command name: PRINT
>The line width specified exceeds the output page width or the record length or
>the maximum record length of 2147483647. Reduce the number of variables or
>split the output line into several records.
>Execution of this command stops.
Try the LIST command.
I often use the TEMPORARY commond prior to the LIST command, as often there is only a small select of record of interest I may want to "list"/investigate.
For example, in the below, only to list the records where VAR2 is not a blank string.
TEMP.
SELECT IF (len(VAR2)>0).
LIST ID VAR2.
Alternatively, you could also (but dependent on having CUSTOM TABLES add-on module), do something like below which would get the results into a tabular format also (which may be preferable if then exporting to Excel, for example.
CTABLES /TABLE CTABLES /VLABELS VARIABLES=ALL DISPLAY=NONE
/TABLE A[C]>B[C]
/CATEGORIES VARIABLES=ALL EMPTY=EXCLUDE.