Heroku not listing pg_search_documents table - ruby-on-rails

This is pretty weird but for some reason, heroku doesn't seem to show the pg_search_documents table when when I list tables using the heroku-sql-console.
>> heroku sql
SQL> show tables
+------------------------+
| table_name |
+------------------------+
| activity_notifications |
| attachments |
| businesses |
| color_modes |
| comments |
| counties |
| customer_employees |
| customers |
| delayed_jobs |
| file_imports |
| invitations |
| invoices |
| jobs |
| paper_stocks |
| paper_weights |
| quotes |
| rails_admin_histories |
| schema_migrations |
| tax_rates |
| users |
+------------------------+
As you can see, no mention of pg_search.
Then, in the same session,
SQL> select * from pg_search_documents;
+---------------------------------------------------------------------------------------------------------------------+
| id | content | searchable_id | searchable_type | created_at | updated_at |
+---------------------------------------------------------------------------------------------------------------------+
| 3 | Energy Centre | 3 | Customer | 2012-12-03 19:33:55 -0800 | 2012-12-03 19:33:55 -0800 |
+---------------------------------------------------------------------------------------------------------------------+
It's also interesting that the show tables command lists only 20 tables whereas heroku pg:info says there are 21.
The reason this is a problem rather than a curiosity is because I can't get heroku db:pull to pull down the pg_search_documents table (everything else pulls fine) and I can't test migrations on production data.
I'm using PG Version: 9.1.6 on heroku and PostgreSQL 9.2.1 locally. Also PgSearch 0.5.7.
Any ideas what the issue is?

Related

Neo4j Cypher: How to optimize a NOT EXISTS Query when cardinality is high

The below query takes over 1 second & consumer about 7 MB when cardinality b/w users to posts is about 8000 (one user views about 8000 posts). It is difficult to scale this due to high & linearly growing latencies & memory consumption. Is there a possibility to model this differently and/or optimise the query?
Query
PROFILE MATCH (u:User)-[:CREATED]->(p:Post) WHERE NOT (:User{ID: 2})-[:VIEWED]->(p) RETURN p.ID
Plan
| Plan | Statement | Version | Planner | Runtime | Time | DbHits | Rows | Memory (Bytes) |
+-----------------------------------------------------------------------------------------------------------+
| "PROFILE" | "READ_ONLY" | "CYPHER 4.1" | "COST" | "INTERPRETED" | 1033 | 3721750 | 10 | 6696240 |
+-----------------------------------------------------------------------------------------------------------+
+------------------------------+-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| Operator | Details | Estimated Rows | Rows | DB Hits | Cache H/M | Memory (Bytes) | Ordered by |
+------------------------------+-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| +ProduceResults#neo4j | `p.ID` | 2158 | 10 | 0 | 0/0 | | |
| | +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| +Projection#neo4j | p.ID AS `p.ID` | 2158 | 10 | 10 | 0/0 | | |
| | +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| +Filter#neo4j | u:User | 2158 | 10 | 10 | 0/0 | | |
| | +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| +Expand(All)#neo4j | (p)<-[anon_15:CREATED]-(u) | 2158 | 10 | 20 | 0/0 | | |
| | +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| +AntiSemiApply#neo4j | | 2158 | 10 | 0 | 0/0 | | |
| |\ +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| | +Expand(Into)#neo4j | (anon_47)-[anon_61:VIEWED]->(p) | 233 | 0 | 3695819 | 0/0 | 6696240 | anon_47.ID ASC |
| | | +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| | +NodeUniqueIndexSeek#neo4j | UNIQUE anon_47:User(ID) WHERE ID = $autoint_0 | 8630 | 8630 | 17260 | 0/0 | | anon_47.ID ASC |
| | +-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
| +NodeByLabelScan#neo4j | p:Post | 8630 | 8630 | 8631 | 0/0 | | |
+------------------------------+-----------------------------------------------+----------------+------+---------+-----------+----------------+----------------+
Yes, this can be improved.
First, let's understand what this is doing.
First, it starts with a NodeByLabelScan. That makes sense, there's no avoiding that.
But then, for every node of the label (the following executes PER ROW!), it matches to user 2, and expands all :VIEWED relationships from user 2 to see if any of them is the post for that particular row.
Can you see why this is inefficient? There are 8630 post nodes according to the PROFILE plan, so user 2 is looked up by index 8630 times, and their :VIEWED relationships are expanded 8630 times. Why 8630 times? Because this is happening per :Post node.
Instead, try this:
MATCH (:User{ID: 2})-[:VIEWED]->(viewedPost)
WITH collect(viewedPost) as viewedPosts
MATCH (:User)-[:CREATED]->(p:Post)
WHERE NOT p IN viewedPosts
RETURN p.ID
This changes things up a bit.
First it matches to user 2's viewed posts (the lookup and expansion is performed only once), then those viewed posts are collected.
Then it will do a label scan, and filter such that the post isn't in the collection of viewed posts.

YouTube API v3 returning 400 error when using pageToken

I am trying to get paginated comments on a video using following request:
https://www.googleapis.com/youtube/v3/commentThreads?part=snippet,replies&videoId=zB8byQHNHHg&maxResults=5&pageToken='2'&key=randomstring
But it is returning following response which shows 400 error:
What could be the reason as the official documentation states using pageToken for doing the same.
https://developers.google.com/youtube/v3/docs/commentThreads/list#parameters
Anyone solved similar issue in the past? Please help.
The following table list the page along with the corresponding page token and total results. Note that you do not (need to) pass in a pageToken for the first page.
+------+------------+---------------+
| Page | Page Token | Total Results |
+------+------------+---------------+
| 1 | [none] | 3628 |
| 2 | CDIQAA | 3628 |
| 3 | CGQQAA | 3626 |
| 4 | CJYBEAA | 3626 |
| 5 | CMgBEAA | 3625 |
| 6 | CPoBEAA | 3625 |
| 7 | CKwCEAA | 3625 |
| 8 | CN4CEAA | 3624 |
| 9 | CJADEAA | 3624 |
| 10 | CMIDEAA | 3624 |
+------+------------+---------------+

InfluxDB select different time between two row having same a field value

I have a table like this on InfluxDB:
+---------------+-----------------+--------+--------------------------+
| time | sequence_number | action | session_id |
+---------------+-----------------+--------+--------------------------+
| 1433322591220 | 270001 | delete | 556d85bfe26c3b3864617605 |
| 1433322553324 | 250001 | delete | 556d88e4e26c3b3b83c99d32 |
| 1433241828472 | 230001 | create | 556d88e4e26c3b3b83c99d32 |
| 1433241023633 | 80001 | create | 556d85bfe26c3b3864617605 |
| 1433239305306 | 70001 | create | 556d7f09e26c3b34e872b2ba |
+---------------+-----------------+--------+--------------------------+
Now I want to find the time range from a session be created to deleted, that means get time where action=delete minus time where action=create if they have same session_id

find association by name

I have a User model that has_many friends. I want to be able to find all the friends of a user by name.
I want to be able to find all users with the string "john" in their username.
Users
id | email | created_at | updated_at | password_digest | remember_token | admin | username | password_reset_token | password_reset_sent_at | provider | uid | generated_password
-------+---------------------------+----------------------------+----------------------------+--------------------------------------------------------------+------------------------+-------+---------------+----------------------+------------------------+----------+-----+--------------------
27288 | ******#hotmail.com | 2014-02-19 17:25:22.545866 | 2014-02-19 17:25:22.545866 | $2a$10$hjdAPWYTorCN7NW6QawmtOMqTk.DP8m2e.I9XjfujRG8em8eq6z9G | dxowM5tTTalNLWSO_5j8sw | f | huntingtonl93 | | | | | f
23 | *******#gmail.com | 2013-07-05 17:18:59.64622 | 2013-07-05 17:18:59.64622 | $2a$10$4D0cML22vnVYuT0tJTJwMO/T4xuiGYUU70BnK1ug3K.TE1.pWG64e | FS2aVM57bX0ZiMBNuWnxLA | f | one | | | | | f
31 | ********#gmail.com | 2013-09-06 21:47:52.869246 | 2013-09-06 21:47:52.869246 | $2a$10$XW6jPGZTiYo6KZm4WWOBye0b.mpNuKGt89gMLVRCjF7kjuhzHXzJG | TUsweMN36i_eouveEFU9cQ | f | drebenok | | | | | f
6388 | *******#startupv8.com | 2013-11-25 00:20:44.124276 | 2013-11-25 00:20:44.124276 | $2a$10$O5V/8C2vs.CpkYDDwByjiOOVkf0FlWb8lsMVX/r8cPnHbL8G0gRda | 2duwh0WURFI10I1PcBnD9Q | f | ihab
Friends
id | user_id | user_id_friend | created_at | updated_at
----+---------+----------------+----------------------------+----------------------------
10 | 15 | 23 | 2013-08-13 01:55:41.406368 | 2013-08-13 01:55:41.406368
12 | 15 | 28 | 2013-08-13 01:55:57.025223 | 2013-08-13 01:55:57.025223
14 | 16 | 23 | 2013-08-13 01:57:35.910647 | 2013-08-13 01:57:35.910647
16 | 17 | 23 | 2013-08-13 01:58:01.051038 | 2013-08-13 01:58:01.051038
18 | 17 | 24 | 2013-08-13 01:58:08.866769 | 2013-08-13 01:58:08.866769
You can do this by adding conditions to your association like so:
user.friends.where("username like ?", '%john%')
This involves just a little bit of SQL knowledge. We want the username to be "like" the string we give it. The string itself ('%john%') will match any username that contains "john" somewhere inside. The "%" is a wildcard character that will match any characters that might come before or after "john".
If you only wanted to find users whose usernames start with john, you would do it like so:
user.friends.where("username like ?", "john%")
If you only wanted to find users whose usernames end with john, you do this instead:
user.friends.where("username like ?", "%john")
This approach can be applied to any conditions you would use when searching in ActiveRecord. For instance, you could find all of a users's friends who don't have e-mail addresses:
user.friends.where(email: nil)
I hope this helps!
You can quite easily do it with gem "Ransack". It allows you to perform searches like that with ease.
https://github.com/activerecord-hackery/ransack

Access violation while the program was idle - not trace information to track down the bug

I have a program that just popped up an AV. Until now the Eureka Log could find the source code line that generated the error but now it displays only this:
Access violation at address 7E452E4E in module 'USER32.dll'. Read of address 00000015.
Call Stack Information:
--------------------------------------------------------------------------------------------
|Address |Module |Unit |Class|Procedure/Method |Line |
--------------------------------------------------------------------------------------------
|Running Thread: ID=2640; Priority=0; Class=; [Main] |
|------------------------------------------------------------------------------------------|
|77F16A7E|GDI32.dll | | |IntersectClipRect | |
|7E433000|USER32.dll | | |EditWndProc | |
|7E42A993|USER32.dll | | |CallWindowProcA | |
|7E42A97D|USER32.dll | | |CallWindowProcA | |
|7E429011|USER32.dll | | |OffsetRect | |
|7E4196C2|USER32.dll | | |DispatchMessageA | |
|7E4196B8|USER32.dll | | |DispatchMessageA | |
|00625E13|Amper.exe |Amper.DPR | | |76[16]|
|7C915511|ntdll.dll | | |RtlFindActivationContextSectionString| |
|7C915D61|ntdll.dll | | |RtlFindCharInUnicodeString | |
|7C910466|ntdll.dll | | |RtlFreeUnicodeString | |
|7C80B87C|kernel32.dll | | |IsDBCSLeadByte | |
|7C9113ED|ntdll.dll | | |RtlDeleteCriticalSection | |
|7C80EEF5|kernel32.dll | | |FindClose | |
|7C901000|ntdll.dll | | |RtlEnterCriticalSection | |
|7C912CFF|ntdll.dll | | |LdrLockLoaderLock | |
|7C9010E0|ntdll.dll | | |RtlLeaveCriticalSection | |
|7C912D19|ntdll.dll | | |LdrUnlockLoaderLock | |
|7C9166C1|ntdll.dll | | |LdrGetDllHandleEx | |
|7C9166B3|ntdll.dll | | |LdrGetDllHandle | |
|7C9166A0|ntdll.dll | | |LdrGetDllHandle | |
|7C912A8D|ntdll.dll | | |RtlUnicodeToMultiByteN | |
|7C912C21|ntdll.dll | | |RtlUnicodeStringToAnsiString | |
|7C901000|ntdll.dll | | |RtlEnterCriticalSection | |
|7C912CC9|ntdll.dll | | |LdrLockLoaderLock | |
|7C912CFF|ntdll.dll | | |LdrLockLoaderLock | |
|7C9010E0|ntdll.dll | | |RtlLeaveCriticalSection | |
|7C912D19|ntdll.dll | | |LdrUnlockLoaderLock | |
|7C90CF78|ntdll.dll | | |ZwAllocateVirtualMemory | |
|7C90CF6E|ntdll.dll | | |ZwAllocateVirtualMemory | |
|7C9010E0|ntdll.dll | | |RtlLeaveCriticalSection | |
|7C80BA57|kernel32.dll | | |VirtualQueryEx | |
|7C80BA40|kernel32.dll | | |VirtualQueryEx | |
|7C80BA81|kernel32.dll | | |VirtualQuery | |
|7C901000|ntdll.dll | | |RtlEnterCriticalSection | |
|7C912CC9|ntdll.dll | | |LdrLockLoaderLock | |
|7C912CFF|ntdll.dll | | |LdrLockLoaderLock | |
|7C9010E0|ntdll.dll | | |RtlLeaveCriticalSection | |
--------------------------------------------------------------------------------------------
The program was totally idle while I got the error and its window was hidden by other windows. FastMM is active and set to full debug but it indicates no memory overwrite.
Any hints about how to find the origin of this AV?
Win XP, Delphi 7
I don't see an EditWndProc() method in user32.dll, but Delphi has a couple -- one dealing with combobox messages and one dealing with tree views. Given MS's comctrl mess, I'd guess you have a tree view?
Check your tree view stuff. Given IntersectClipRect's parameters, it's easy to guess that it's being passed an invalid device context -- so...are you doing any custom painting for your tree view? If so, are you checking to make sure the canvas handle is ! NIL before you begin painting (try assertions if nothing else)?
I just wonder what's on line 76[16] in Amper.exe... That line number might be a hint of the location of the error.
Then again, when it's just happening during an idle moment then it basically happens when the system is processing Windows messages like the mouse moving, keyboard events, timer updates and a lot more.
It sometimes helps to search for the error message plus code. I've done a quick scan and found this KB from MS which suggests that this kind of error can happen when you call certain Windows API's with invalid parameters. But this KB doesn't apply to your error. Still, it gives you an idea about what to check: any Windows API call you make in your own code.
Does it also generate this exception in the IDE, while you're debugging?
That's what EurekaLog does when it has nothing to work with. You need to rebuild and have the linker produce a detailed map file. That's how it knows what to apply its stack trace to.

Resources