Way to load mvcc tables automatically when DolphinDB starts? - startup

I create the following 3 mvcc tables
n=5
syms=`IBM`C`MS`MSFT`JPM`ORCL`FB`GE
timestamp=09:30:00+rand(18000,n)
sym=rand(syms,n)
qty=100*(1+rand(100,n))
price=5.0+rand(100.0,n)
temp=table(timestamp,sym,qty,price)
t1= mvccTable(1:0,`timestamp`sym`qty`price,[TIMESTAMP,SYMBOL,INT,DOUBLE],"/home/xjqian/mvcctables","t1")
t1.append!(temp);
t2= mvccTable(1:0,`timestamp`sym`qty`price,[TIMESTAMP,SYMBOL,INT,DOUBLE],"/home/xjqian/mvcctables","t2")
t2.append!(temp);
t3= mvccTable(1:0,`timestamp`sym`qty`price,[TIMESTAMP,SYMBOL,INT,DOUBLE],"/home/xjqian/mvcctables","t3")
t3.append!(temp);
Is there any way to automatically load all these tables when DolphinDB is started?

The script can be used to get all table names and load the tables:
login(`admin,`123456)
tables=select * from files("/home/xjqian/mvcctables") where isDir=true
for (tableName in tables.filename){
share( loadMvccTable("/home/xjqian/mvcctables",tableName) , "share_"+tableName)
}
Add it to the startup script startup.dos, so the tables can be loaded when the system is started. See the Startup Scripts for more information.
You can check whether the tables are loaded with objs(true):
select * from objs(true) where name like "share_%"
See the queried result:
name type form rows columns bytes shared extra
share_t1 MVCC TABLE 5 4 368 true
share_t2 MVCC TABLE 5 4 368 true
share_t3 MVCC TABLE 5 4 368 true

Related

Small Lookup Join Pyspark

In order to build a logic in one of Pyspark code , I have to join distinct values in a big data frame with a small lookup table . What could be the recommended way to execute the same . Is there any quick way to join this with out involving parallel processing or broadcast
df_distinct = Main Data Frame - 10 Million records -- After Distinct ~ around 100 records
dflookup = Look up table - 250 records
df_join= Output ~ 100 records

Grails maximum number of columns for a table

There is some strange limit into grails,gorm about the maximum number of columns for a domain class (table)?
In the class definition with 192 columns hibernate create succesfully the table during application boot,with 193 no table is created and no errors in the application log

Informix Query Tuning

I have a table called lead, which have about 500 thousand records and we need the following query to get executed.
SELECT skip 300000 first 75 *
FROM lead
WHERE ((enrollment_period IS NULL) OR
(enrollment_period IN ('FT2015','F16','SUM2016','FALL2016','FALL2017','SP17')))
ORDER BY created_on DESC
The table lead has id column as the primary key and thus have clustered index in that column. This query is taking about 12 - 13 mins. When I added a non-clustered index on created_on and enrollment_period columns, it came down to 4 - 5 mins. Then I changed the clustered index from id column to this index, execution time came down further to about 50 seconds now.
Is there any other optimization scope available for this query?
Overall, is there any other change that can be done so that the query will execute faster?
Thanks in Advance,
Manohar

Delphi Table sorting

I have a simple problem which gave em a headache
I need to sort integers in a Database table TDBGrid ( its ABS database from component ace ) with the following order
0
1
11
111
121
2
21
211
22
221
and so on
which means every number starting with 1 should be under 1
1
11
111
5
55
can anyone help me?
thanks
This should work to get stuff in the right order:
Convert the original number to a string;
Right-pad with zeroes until you have a string of 3 characters wide;
(optional) Convert back to integer.
Then sorting should always work the way you want. Probably it's best to let the database do that for you. In MySQL you'd do something like this:
select RPAD(orderid,3,'0') as TheOrder
from MyTable
order by 1
I just ran this in SQL Server Management Studio - note I mixed up the rows in the input so they were not in sorted order:
create table #temp( ID Char(3));
insert into #temp (ID)
select '111' union
select '221';
select '0' union
select '21' union
select '1' union
select '11' union
select '211' union
select '121' union
select '2' union
select '22' union
select * from #temp order by ID;
I got the following output:
ID
----
0
1
11
111
121
2
21
211
22
221
(10 row(s) affected)
If you're getting different results, you're doing something wrong. However, it's hard to say what because you didn't post anything about how you're retrieving the data from the database.
Edit: Some clarification by the poster indicates that the display is in a TDBGrid attached to a table using Component Ace ABS Database. If that indeed is the case, then the answer is to create an index on the indicated column, and then set the table's IndexName property to use that index.
select cast(INT_FIELD as varchar(9)) as I
from TABxxx
order by 1

350+ errors: The type 'blah.blah.blah' already contains a definition?

What does this mean. Is it because I have two different .DBML files that contain the same database table?
...
Error 343 The type 'mvc.Models.Bundle' already contains a definition for 'BundleIcon' C:\inetpub\wwwroot\Models\Assets1.designer.cs 3438 17 mvc
Error 344 The type 'mvc.Models.Bundle' already contains a definition for 'isScorm' C:\inetpub\wwwroot\Models\Assets1.designer.cs 3459 15 mvc
Error 345 The type 'mvc.Models.Bundle' already contains a definition for 'scormPath' C:\inetpub\wwwroot\Models\Assets1.designer.cs 3480 17 mvc
Error 346 The type 'mvc.Models.Bundle' already contains a definition for 'CompanyID' C:\inetpub\wwwroot\Models\Assets1.designer.cs 3501 14 mvc
...
Yes, if you keep them in the same namespace this would occurr.
OK, I came across this same error when adding another Linq-to-SQL .dbml.
The more specific cause is that you can not have 2 separate .DBML's in the same namespace which reference the same table and column.
Unlike in Datasets, wherein you can have 2 separate Datasets (Dataset1.xsd and Dataset2.xsd) weach ith the same table and same columns, not so in Linq.
DataClass1.dbml with table MyTable with a column myColumn and DataClass2.dbml with table also name MyTable with a column myColumn will fail because myColumn is defined in both designer.cs files within the same namespace.
My workaround: I 'renamed' DataClass2.dbml's MyTable to MyTable_2 and myColumn to myColumn_2.
I then cursed Microsoft, deleted DataClass2.dbml and integrated the 3rd table I needed into DataClass1.dbml, along with other tables (to avoid this issue). DataClass1.dbml now has some 40 tables in it, which now causes the DataClass1.designer.cs file to have over 20,000 lines of 'auto-generated' code.
Lovely, eh.

Resources