events-aggregator-berlin

A Ruby on Rails application to aggregate events in Berlin from different sources

Walkthrough

This app uses postgres database.

1. After creating the database and running migrations, run rails db:seed since the seed file contains WebSources which contains information about the sites that we are going to scrape.

   app/services/scraping_service.rb contains the code for scraping!

2. Go to rails console, and run the scrapingservice for initial data.
   s = ScrapingService.new
s.perform
   This could also be placed inside seeds.rb.

3. This app uses sidekiq to run ScrapingService in the background. For scheduling the job every 30 minutes (or 1 hour), we use sidekiq-scheduler.
   config/sidekiq.yml contains the schedule for running the service.

In order to run sidekiq, you can open up a terminal window and run sidekiq. Ideally, you should have redis installed before since sidekiq uses redis.
After sidekiq runs, you can go to http://localhost:3000/sidekiq/busy to see the jobs in queue or go to the dashboard page for overall stats. This is possible because we are using the sidekiq web UI.

Kaminari is used for pagination. I have edited _paginator.html.erb to remove numbers and instead relied only on First, Next, Last and Previous buttons.

Bootstrap is used for overall styling. The styling is pretty basic and there are tonnes of things that can be improved here! I have used the card layout.

I have used RSpec for testing. spec/rails_helpers.rb contains the configuration for shoulda-matchers, capybara and Poltergeist. I have used capybara, Poltergeist, and phantomjs for testing http requests and verifying that they have the required content.

This resource has proved to be helpful: https://medium.com/@ethanryan/testing-your-app-in-the-browser-with-capybara-rails-backend-react-frontend-e409671c4596

I have used simplecov and simplecov-rcov to report the test coverage of my codebase. This can be viewed whenever you run any spec like so:
rspec spec/models/web_source_spec.rb
or
rspec spec/models/event_spec.rb
The static page generated by simplecov is present in app/coverage/index.html for direct viewing.

The tests cover all the model validatons (presence and length checks). The http requests tests to our target websites could further be improved!