Slow Sidekiq in development only - ruby-on-rails

Running Sidekiq 6.1.0 on Rails 5.0.7.2 with Sidekiq concurrency set to 12, Redis server 4.0.14 and Redis gem 4.2.1.
There seems to be some delay (seconds) between queueing/executing jobs that we didn't see with Rails 4.2.9. Note, this behavior only occurs in development, production seems to do just fine.
An example worker:
class SidekiqTestWorker
include Sidekiq::Worker
sidekiq_options(
queue: "default",
)
def perform
puts "Hello from Sidekiq!"
end
end
Running 1000.times { SidekiqTestWorker.perform_async } in a Rails console takes around a second to execute all jobs with Rails 4.2.9, but with Rails 5.0.7.2 it takes several minutes to complete. Worth mentioning that we tried running with the same Sidekiq version (5.2.8) with only Rails diffing between the tries with the same result.
A snippet from the Sidekiq worker logs shows the behavior (note the timestamp):
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:00 sidekiq_worker.1 | Hello from Sidekiq!
16:52:01 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:04 sidekiq_worker.1 | Hello from Sidekiq!
16:52:05 sidekiq_worker.1 | Hello from Sidekiq!
16:52:05 sidekiq_worker.1 | Hello from Sidekiq!
16:52:05 sidekiq_worker.1 | Hello from Sidekiq!
Any ideas what can be done to fix this?

In the Sidekiq 5.0 release notes:
Sidekiq 5.0 contains a reworked job dispatch and execution core to integrate better with the new Rails 5.0 Executor.
The Rails Executor is single-threaded in development mode so it can hot reload job code changes. Sidekiq can only execute one job at a time.
The only way to fix this is to enable eager loading in config/environment/development.rb but that will disable code reloading too.

Related

Fastlane upload to App Store fails in non-interactive mode even with API key

I am using Fastlane in a GitLab CI environment. I am using an API key and have successfully uploaded my app to TestFlight. However, it fails when uploading the app to the App Store. The build uploads successfully, but it fails when waiting for a response after sending the metadata. Here are the relevant snippets:
# Fastfile
lane :get_app_store_connect_api_key do
ensure_env_vars(
env_vars: [
'APP_STORE_CONNECT_API_KEY'
]
)
app_store_connect_api_key(
key_id: "**********",
issuer_id: "********-****-****-****-************",
key_content: "#{ENV["APP_STORE_CONNECT_API_KEY"]}".gsub('\n', '\\n'),
in_house: false,
)
end
...
desc "Push a new release channel build to the App Store."
lane :release do
build_release()
appstore(
api_key: get_app_store_connect_api_key
)
end
LOG
[17:22:47]: ----------------------
[17:22:47]: --- Step: appstore ---
[17:22:47]: ----------------------
[17:22:47]: Successfully loaded '/Users/********/builds/qVPSw-s1/0/********/********/client/ios/fastlane/Deliverfile' ๐Ÿ“„
[17:22:47]: No values defined in './fastlane/Deliverfile'
[17:22:47]: Creating authorization token for App Store Connect API
+--------------------------------------+------------------------+
| deliver 2.181.0 Summary |
+--------------------------------------+------------------------+
| api_key | ******** |
| screenshots_path | ./fastlane/screenshots |
| metadata_path | ./fastlane/metadata |
| app_version | 1.1.0 |
| username | ******** |
| app_identifier | com.********.app |
| ipa | Runner.ipa |
| platform | ios |
| edit_live | false |
| use_live_version | false |
| skip_binary_upload | false |
| skip_screenshots | false |
| skip_metadata | false |
| skip_app_version_update | false |
| force | false |
| overwrite_screenshots | false |
| submit_for_review | false |
| reject_if_possible | false |
| phased_release | false |
| reset_ratings | false |
| team_id | ********|
| dev_portal_team_id | ********|
| run_precheck_before_submit | true |
| precheck_default_rule_level | warn |
| ignore_language_directory_validation | false |
| precheck_include_in_app_purchases | true |
+--------------------------------------+------------------------+
[17:22:52]: Making sure the latest version on App Store Connect matches '1.1.0'...
[17:22:54]: Successfully set the version to '1.1.0'
[17:22:54]: Loading './fastlane/metadata/en-US/description.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/keywords.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/release_notes.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/support_url.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/marketing_url.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/promotional_text.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/name.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/subtitle.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/privacy_url.txt'...
[17:22:54]: Loading './fastlane/metadata/en-US/apple_tv_privacy_policy.txt'...
[17:22:54]: Loading './fastlane/metadata/copyright.txt'...
[17:22:54]: Loading './fastlane/metadata/primary_category.txt'...
[17:22:54]: Loading './fastlane/metadata/secondary_category.txt'...
[17:22:54]: Loading './fastlane/metadata/primary_first_sub_category.txt'...
[17:22:54]: Loading './fastlane/metadata/primary_second_sub_category.txt'...
[17:22:54]: Loading './fastlane/metadata/secondary_first_sub_category.txt'...
[17:22:54]: Loading './fastlane/metadata/secondary_second_sub_category.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/first_name.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/last_name.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/phone_number.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/email_address.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/demo_user.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/demo_password.txt'...
[17:22:54]: Loading './fastlane/metadata/review_information/notes.txt'...
[17:22:54]: Verifying the upload via the HTML file can be disabled by either adding
[17:22:54]: `force true` to your Deliverfile or using `fastlane deliver --force`
[17:22:57]: Does the Preview on path './fastlane/Preview.html' look okay for you?
+------------------------------------+-----------------------------------------------------------------------------------------------------+
| Lane Context |
+------------------------------------+-----------------------------------------------------------------------------------------------------+
| DEFAULT_PLATFORM | ios |
| PLATFORM_NAME | ios |
| LANE_NAME | ios release |
| SIGH_PROFILE_TYPE | app-store |
| MATCH_PROVISIONING_PROFILE_MAPPING | {"com.********.app"=>"match AppStore com.********.app"} |
| IPA_OUTPUT_PATH | /Users/********/builds/qVPSw-s1/0/********/********/client/ios/Runner.ipa |
| XCODEBUILD_ARCHIVE | /Users/********/Library/Developer/Xcode/Archives/2021-04-21/Runner 2021-04-21 17.14.18.xcarchive |
| DSYM_OUTPUT_PATH | /Users/********/builds/qVPSw-s1/0/********/********/client/ios/Runner.app.dSYM.zip |
+------------------------------------+-----------------------------------------------------------------------------------------------------+
[17:22:57]: Could not retrieve response as fastlane runs in non-interactive mode
+------+--------------------------------------------------+-------------+
| fastlane summary |
+------+--------------------------------------------------+-------------+
| Step | Action | Time (in s) |
+------+--------------------------------------------------+-------------+
| 1 | default_platform | 0 |
| 2 | Switch to ios build_release lane | 0 |
| 3 | Switch to ios certificate_release lane | 0 |
| 4 | Switch to ios get_app_store_connect_api_key lane | 0 |
| 5 | ensure_env_vars | 0 |
| 6 | app_store_connect_api_key | 0 |
| 7 | match | 2 |
| 8 | gym | 522 |
| 9 | Switch to ios get_app_store_connect_api_key lane | 0 |
| 10 | ensure_env_vars | 0 |
| 11 | app_store_connect_api_key | 0 |
| ๐Ÿ’ฅ | appstore | 9 |
+------+--------------------------------------------------+-------------+
[17:22:57]: fastlane finished with errors
Looking for related GitHub issues on fastlane/fastlane...
โžก๏ธ Metadata Path is not considered
https://github.com/fastlane/fastlane/issues/18321 [open] 4 ๐Ÿ’ฌ
2 weeks ago
โžก๏ธ GITLAB CI: Could not retrieve response as fastlane runs in non-interactive mode
https://github.com/fastlane/fastlane/issues/17376 [closed] 18 ๐Ÿ’ฌ
17 Feb 2021
โžก๏ธ FastlaneCore::Interface::FastlaneCrash: [!] Could not retrieve response as fastlane runs in non-interactive mode
https://github.com/fastlane/fastlane/issues/17488 [closed] 11 ๐Ÿ’ฌ
16 Jan 2021
and 53 more at: https://github.com/fastlane/fastlane/search?q=Could%20not%20retrieve%20response%20as%20fastlane%20runs%20in%20non-interactive%20mode&type=Issues&utf8=โœ“
๐Ÿ”— You can โŒ˜ + double-click on links to open them directly in your browser.
bundler: failed to load command: fastlane (/Users/********/builds/qVPSw-s1/0/********/********/client/ios/vendor/bundle/ruby/2.6.0/bin/fastlane)
/Users/********/builds/qVPSw-s1/0/********/********/client/ios/vendor/bundle/ruby/2.6.0/gems/fastlane-2.181.0/fastlane_core/lib/fastlane_core/ui/interface.rb:129:in `crash!': [!] Could not retrieve response as fastlane runs in non-interactive mode (FastlaneCore::Interface::FastlaneCrash)
Which access has your API key? Try to create a new key with App Manager access.
I was able to fix this in my project by adding the force: true option to the upload_to_appstore call in our fastfile. The Fastlane docs list force as an option, but it's hidden behind a "More Options" dropdown.
Also like the other answer says you need to have the proper permissions for your API key. You need an API Key with App Manager permissions to create a new build, which also lets you upload the ipa. I read somewhere that Developer permission is enough to update the build after an App Manager creates it, however I haven't actually tried it.
The lane in our fastfile now looks like this:
lane :release_appstore do
upload_to_app_store(
ipa: "<ipa_path>.ipa",
api_key_path: "<api_key_path>.json",
force: true
)
end

Jenkins failure build: Internal Error: Unhandled kpi type with performance plugin

I'm running Jenkins with the Performance Plugin.
I have a multiple JMeter jmx scripts which run on jenkins and I'm trying to add this one. But the build is always failing. With this message
Internal Error: Unhandled kpi type: <type 'long'>
bzt installation is also done.
I can't seem to find much info about this on google. Any help?
After the shutdown I'm getting this:
19:42:27 INFO: Shutting down...
19:42:27 INFO: Post-processing...
19:42:29 INFO: Test duration: 0:47:04
19:42:29 INFO: Samples count: 3200, 3.25% failures
19:42:29 INFO: Average times: total 4.369, latency 0.000, connect 0.000
19:42:29 INFO: Percentiles:
+---------------+---------------+
| Percentile, % | Resp. Time, s |
+---------------+---------------+
| 0.0 | 0.258 |
| 50.0 | 3.251 |
| 90.0 | 8.799 |
| 95.0 | 14.375 |
| 99.0 | 24.239 |
| 99.9 | 30.031 |
| 100.0 | 35.743 |
+---------------+---------------+
19:42:29 INFO: Request label stats:
+----------------------+--------+--------+--------+---------------------------------------------------------------------+
| label | status | succ | avg_rt | error |
+----------------------+--------+--------+--------+---------------------------------------------------------------------+
| Click_ToonSelectie | FAIL | 94.00% | 4.984 | Number of samples in transaction : 4, number of failing samples : 1 |
| | | | | Non HTTP response message: Connection timed out: connect |
| | | | | Not Modified |
| FilterBrand | FAIL | 99.00% | 0.832 | Number of samples in transaction : 1, number of failing samples : 1 |
| LoadFilterPage | FAIL | 96.00% | 6.306 | Non HTTP response message: Connection timed out: connect |
| | | | | Number of samples in transaction : 3, number of failing samples : 1 |
| OpenRandomCarDetails | FAIL | 96.00% | 5.355 | Number of samples in transaction : 1, number of failing samples : 1 |
| | | | | Moved Permanently |
+----------------------+--------+--------+--------+---------------------------------------------------------------------+
19:42:29 INFO: Request label stats:
+----------------------+--------+--------+--------+---------------------------------------------------------------------+
| label | status | succ | avg_rt | error |
+----------------------+--------+--------+--------+---------------------------------------------------------------------+
| Click_ToonSelectie | FAIL | 94.00% | 4.984 | Number of samples in transaction : 4, number of failing samples : 1 |
| | | | | Non HTTP response message: Connection timed out: connect |
| | | | | Not Modified |
| FilterBrand | FAIL | 99.00% | 0.832 | Number of samples in transaction : 1, number of failing samples : 1 |
| LoadFilterPage | FAIL | 96.00% | 6.306 | Non HTTP response message: Connection timed out: connect |
| | | | | Number of samples in transaction : 3, number of failing samples : 1 |
| OpenRandomCarDetails | FAIL | 96.00% | 5.355 | Number of samples in transaction : 1, number of failing samples : 1 |
| | | | | Moved Permanently |
+----------------------+--------+--------+--------+---------------------------------------------------------------------+
19:42:29 INFO: Dumping final status as XML: aggregate-results.xml
19:42:29 ERROR: Internal Error: Unhandled kpi type: <type 'long'>
19:42:29 INFO: Artifacts dir: C:\Users\Kristof\.jenkins\workspace\ACC-Tweedehands\2018-07-19_18-55-13.298000
19:42:29 WARNING: Done performing with code: 1
Build step 'Run Performance Test' changed build result to FAILURE
Finished: FAILURE
First 2 lines of JTL:
<?xml version="1.0" encoding="UTF-8"?>
<testResults version="1.2">
If you're using Taurus tool my expectation is that you should be providing kpi.jtl CSV file instead of .jtl results file in XML format.
I cannot reproduce your issue using normal kpi.jtl results file from Taurus and latest Performance Plugin version 3.10
I neither can reproduce your issue, my output is totally different:
Started by user anonymous
Building on master in workspace /Users/jenkins/Projects/temp/Jenkins/.jenkins/workspace/Taurus
Performance: Recording JMeterCsv reports '/Users/jenkins/Projects/temp/Jenkins/.jenkins/jobs/Taurus/builds/2/temp/kpi.jtl'
Performance: Parsing JMeter report file '/Users/jenkins/Projects/temp/Jenkins/.jenkins/jobs/Taurus/builds/2/performance-reports/JMeterCSV/kpi.jtl'.
Performance: No threshold configured for making the test unstable
Performance: No threshold configured for making the test failure
Performance: File kpi.jtl reported 0.0% of errors [SUCCESS]. Build status is: SUCCESS
Finished: SUCCESS
See How to Run Taurus with the Jenkins Performance Plugin article for comprehensive information and instructions just in case.

Upgrade Orbeon 3.9 to 4.5

We are upgrading Orbeon 3.9 to 4.5.
And we are using MySQL 6.1 and JBOSS 5.1.
Followed the procedures described in http://wiki.orbeon.com/forms/doc/developer-guide/form-runner/oracle-and-mysql-persistence-layers?pli=1 for upgrade.
I am getting the Form Runner Home just fine (showing the old applications and forms).
And also If I open any new form(which I created after upgrade) from form runner, its perfect, no problem.
But When I try to click on any old form , I am getting error "An error has occurred in Orbeon Forms"
And In Orbeon.log file, I can see the following :
2014-05-08 14:38:40,151 ERROR PageFlowControllerProcessor - error caught {controller: "oxf:/apps/fr/page-flow.xml", method: "GET", path: "/fr/ApplicationName/formName/edit/5273bf03-9f0a-4ab9-b441-94c1dcdffad2"}
2014-05-08 14:38:40,274 ERROR PageFlowControllerProcessor -
+----------------------------------------------------------------------------------------------------------------------+
|An Error has Occurred |
|----------------------------------------------------------------------------------------------------------------------|
|XPath syntax error at char 0 in {}: |
| Unexpected token <"eof"> in path expression
|
|----------------------------------------------------------------------------------------------------------------------|
|Application Call Stack |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/apps/fr/page-flow.xml |executing processor | 208|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|name={http://www.orbeon.com/oxf/processors}pipeline |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/config/epilogue.xpl |executing processor | 60|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|element=[...] |
|name ={}pipeline |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/config/epilogue-servlet.xpl |reading processor output | 31|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|element= |
|name =xformed-data |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/config/epilogue.xpl |reading processor output | 41|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|element= |
|name =xformed-data |
|id =xformed-data |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/ops/pfc/xforms-epilogue.xpl |reading processor output | 95|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|element= |
|name =data |
|ref =xformed-data |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/ops/pfc/xforms-epilogue.xpl |reading processor output | 88|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|element= |
|name =data |
|id =xincluded-data |
|----------------------------------------------------------------------------------------------------------------------|
|oxf:/ops/pfc/xforms-epilogue.xpl |reading processor output | 77|
|ยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยทยท|
|element= |
|name =document |
|id =xhtml-data |

Heroku not listing pg_search_documents table

This is pretty weird but for some reason, heroku doesn't seem to show the pg_search_documents table when when I list tables using the heroku-sql-console.
>> heroku sql
SQL> show tables
+------------------------+
| table_name |
+------------------------+
| activity_notifications |
| attachments |
| businesses |
| color_modes |
| comments |
| counties |
| customer_employees |
| customers |
| delayed_jobs |
| file_imports |
| invitations |
| invoices |
| jobs |
| paper_stocks |
| paper_weights |
| quotes |
| rails_admin_histories |
| schema_migrations |
| tax_rates |
| users |
+------------------------+
As you can see, no mention of pg_search.
Then, in the same session,
SQL> select * from pg_search_documents;
+---------------------------------------------------------------------------------------------------------------------+
| id | content | searchable_id | searchable_type | created_at | updated_at |
+---------------------------------------------------------------------------------------------------------------------+
| 3 | Energy Centre | 3 | Customer | 2012-12-03 19:33:55 -0800 | 2012-12-03 19:33:55 -0800 |
+---------------------------------------------------------------------------------------------------------------------+
It's also interesting that the show tables command lists only 20 tables whereas heroku pg:info says there are 21.
The reason this is a problem rather than a curiosity is because I can't get heroku db:pull to pull down the pg_search_documents table (everything else pulls fine) and I can't test migrations on production data.
I'm using PG Version: 9.1.6 on heroku and PostgreSQL 9.2.1 locally. Also PgSearch 0.5.7.
Any ideas what the issue is?

Unix script to count line between pattern of output

I am trying to write a unix code where I will be to count no of rows between (ie here 2)
two "|DATE and TIME | XXXXXX |"
Is there any method I can use with combination of egrap and wc -l
Output:
|-------------------------------|
|DATE and TIME | XXXXXX |
|-------------------------------|
| 21-NOV-2012 15:56:51 | 1259 |
| 21-NOV-2012 15:56:51 | 1364 |
|-------------------------------|
|DATE and TIME | XXXXXX |
|-------------------------------|
| 21-NOV-2012 16:06:55 | 1259 |
| 21-NOV-2012 16:06:55 | 1364 |
|-------------------------------|
if the expected result is 9 (since there are empty lines in your example, I don't know if they should be counted):
awk '/DATE and TIME/&&!f{f=1;next;}/DATE and TIME/&&f{print x;exit;}f{x++}' file

Resources