Back-end Engineering Articles

I write and talk about backend stuff like Ruby, Ruby On Rails, Databases, Testing, Architecture / Infrastructure / System Design, Cloud, DevOps, Backgroud Jobs, and more...

Twitter:
@daniel_moralesp

2020-02-21

Upgrade Mission - Upgrading a Rails Platform from Rails 3.x to 5.x

Upgrade Mission - Upgrading a Rails Platform from Rails 3.x to 5.x

Arguments:

  • Is a rescue mission to stabilize the app. It will be high, and that's ok
  • Clearly set expectations
  • Is better to have now a quality code base and not to re-factor each 2 years
  • First things first: be familiar with the current code
  • You need valuable code metrics to combat bad code. Basically because I’ll be working with legacy code
  • Before doing any upgrade, the need for understand the current state of the code
  • Next step: create critical tests first
  • This will help new developers to quickly start making changes
  • Its critical to stick to the plan, and be methodical and organized while the project stabilize

  • Rapid screen results in points (3 mins): 
    • README
    • Version
    • Security
    • Data
    • Code
    • Static analysis
    • Unused code
    • Error
  • Questions (3 mins) 
    • Error monitoring
    • Deployment process
  • Upgrade strategy (3 mins) 
    • Explain
  • Plan (3 mins) 
    • Suggested Asana task changes
    • Realistic estimate
  • Begin with the migration
  • Rescue mission: problem? Legacy code and technical debt 
    • Dockerize
    • Package manager
    • CI/CD
    • Documentation for Deploy
    • Reference data and seeds
    • Stabilize project, long term stability
    • Update/create tests, avoid side effects
    • Upgrading

1- Auditing the current state


  • Running locally: Can I run the project locally? Is there documentation? My steps to up both in the two available
  • Most important current methods, Models, Views and Controllers 

    • Important classes: 
      • controllers/api/v1/users/auth_controller.rb
      • controllers/api/v1/users/registrations_controller.rb
      • controllers/api/v1/profiles_controller.rb
      • controllers/api/v1/parking_sessions_controller.rb
      • controllers/api/v1/parking_fines_controller.rb
      • controllers/api/v1/transactions/

2- Version support


  • Ruby version: 2.2.6 (Ended 3 years and 4 months ago https://endoflife.date/ruby). Upgrade to at least 2.6
  • Rails version: 3.2.13 (Ended more than 4 years ago https://endoflife.date/rails). Upgrade to at least 5.2
  • Postgres version: 
  • Gem to catch outdated gems: bundle_outdated 
    • 32 gems approx. outdated: more than 50%
  • Bundle_outdated: sudo docker-compose exec web bundle outdated
  • https://github.com/scoop/bundle_outdated
  • Bundle_report
  • Bundle_report outdated
  • Bundle_report compatibility --rails-version 3.2.22
  • Deprecations run

3- Security vulnerabilities


  • Static analysis. Brakeman gem: advantages 
    • No scanning, directly in source code
    • In any environment
    • Includes web server and DB
    • Report: 
      • file:///home/project_path/security_vulnerabilities.html
  • Other steps: https://github.com/hardhatdigital/rails-security-audit
  • I’m not security expert, but there are some good another steps to follow
  • For now is out of job scope requirements
  • Take care with: search result pages, messages, comments, reviews
  • Most well known attacks: 
    • XSS/Cross-Site Scripting Attack
    • CSRF
    • SQL Injection
    • Clickjacking

4- Data


5- Code


6- Static Analysis


  • Because I'm unfamiliar with the code
  • Application size: (bundler stats gem) (dependencies and codebase) 
    • Dependencies 
      • Declared gems: These gems are listed in your Gemfile. Many of them depend on other gems.
      • Total gems: These gems are all the gems that are used by your application, either directly or indirectly.
      • Execute 
        • $ gem install bundler-stats
        • $ bundle-stats stats
        • $ sudo docker-compose run web bundle exec rake stats
    • Codebase 
  • Rubocop (how idiomatic is the code?): 
  • Reek: measure “code smells” (what areas of the code might be hard to maintain?) 
    • gem install reek -v 1.3.3
    • reek app/models/
    • Executed can highlight things like: 
      • DuplicateMethodCall
      • IrresponsibleModule
      • TooManyStatements
      • NilCheck
      • UncommunicativeVariableName
  • Flog: measures assignments, branches, and calls 
    • Flog reports the most tortured code in an easy to read pain report. The higher the score, the more pain the code is in
    • gem install flog
    • find lib -name app/models/parking_session.rb | xargs flog
  • RubyCritic: Reek + Flay + Flog 

7- Unused code


  • Useful to delete deprecated code. Can be expensive to maintain unused code. Unused code can confuse developers. Gives false negatives in the task of code stabilization 

8- Error monitoring


9- Deployment process



10- Upgrade strategy


  • First things first: 
    • If the goal is to get to Rails 5.2 and we are on Rails 3.2, we need to make sure to first upgrade to 3.3 and then 3.4.
    • Each minor version of Rails provides deprecation warnings for the next version. If we skip versions we find unexpected errors in our app that will be hard to debug. Some people have used a strategy that does many version jumps at once.
    • This radical approach makes sense only when your Rails application is very small
  • General overview before upgrade 
    • How large is their app? How many models does it have? 
      • See Point#6 - Application size
    • How much test coverage does the app have? 
      • See Point #5: test coverage 28.7%, the ideal is to have 60% is crucial to assess risk. Is it negotiable?
    • Have the team tried to upgrade before?
  • 2 main strategies: 
    • Branch strategy: 
      • Dedicated branch
      • All changes there
      • Deploy to staging to manually test everything
      • Good for small projects
    • Dual-boot and small PRs 
      • App runs with 2 different versions at the same time
      • Small Pull-requests
      • Compatible with CI strategies
      • Don’t break anything in test suite
      • More setup, can be slower all the process
  • Gems: 
  • Documentation: here we have the Upgrade Strategy documented: https://docs.google.com/document/d/12SGIpDtJftsV9OVsGwKqPahsglvUf2RlGpUYdiqbwTI/edit

11- Plan & Timeline


  • Best next steps? Refactor or rewrite? Is it salvageable? 9 out of 10 it is
  • Possible Roadmap (It could change according to what’s important to the company)
    • Database backups - 1 day
    • Working dev environment (enhance gist - README) - 1 day
    • Automated CI/CD - 1-2 days
    • Error monitoring - 1 day
    • Security vulnerabilities - 1-2 days
    • Test coverage - to catch any problem: bottleneck ) - 1 week
      • We decided to make or check test only over this layers

  • Upgrade Strategy - 1 week
  • Incremental refactors - It will continue to be done from now on, but the application would be stable 
    • Take care about internal and external users reported bugs (as commented by Daniela in marketing)
    • Small isolated changes
    • Refactoring strategies: 
      • Layer by layer: models, controllers, views
      • Domain by domain: user management, parkings
      • Combo of two: choose one domain and then each layer of it
      • Delete unused code (point #7)
      • Code logistics: branches

12 - Realistic estimate


  • Is a rescue mission to stabilize the app. It will be high, and that's ok
  • Clearly set expectations
  • Is better to have now a quality code base and not to re-factor each 2 years
  • This will help new developers to quickly start making changes
  • Its critical to stick to the plan, and be methodical and organized while the project stabilize
  • Suggestion for future tickets/tasks: take into account the developer point of view :)
  • Best and worst case scenarios 
    • Best case scenario: 2 months
    • Worst case scenario: 3.5 months

13- Keep up to date