rails4.2 heroku cloudfront serving static assets - ruby-on-rails

Question

I have a rails4 app deployed on heroku and using cloudfront as CDN.
Everything works fine in development. In production only the things in the app/assets are served properly, the images in the public folder can't be found.
What should I do to make the public folder assets work? I'd like to serve them from CDN as well if possible, but I'm open to any working solution
Current settings:
production.rb
config.serve_static_files = ENV['RAILS_SERVE_STATIC_FILES'].present? #enabled by heroku default settings
config.assets.compile = false
config.assets.digest = true
config.action_controller.asset_host = 'mycode.cloudfront.net'
config.font_assets.origin = '*'
gemfile
# I'm NOT using asset_sync gem
gem 'font_assets', '~> 0.1.12' #this is needed to make googlefonts work
gem 'rails_12factor', '~> 0.0.3', group :production
UPDATE:
working heroku logs
2016-01-25T19:00:24.624690+00:00 heroku[router]: at=info method=GET path="/500.html" host=appfaskyn.herokuapp.com request_id=d1a637be-b539-429a-9fcd-5f4d0025b562 fwd="50.250.214.91" dyno=web.1 connect=2ms service=5ms status=304 bytes=48
2016-01-25T19:00:24.794605+00:00 heroku[router]: at=info method=GET path="/thirdlogo.png" host=appfaskyn.herokuapp.com request_id=41b8aa45-8e70-43ef-af53-b4e0ad44ec8b fwd="50.250.214.91" dyno=web.1 connect=0ms service=4ms status=304 bytes=48
UPDATE2:
Realized that it works w/ certain routes.
So if I hit app/dsfd then it finds the pic with path="/dsfd"
but if I hit app/users/2 (user does not exist anymore, so should be the same page does not exist error) then path="/users/2" can't find the pic.
log if found:
2016-01-29T18:56:14.377420+00:00 heroku[router]: at=info method=GET path="/dsfd" host=www.faskyn.com request_id=76e1c73d-dd22-446f-89b4-096dcbae6f6c fwd="64.245.0.218" dyno=web.1 connect=0ms service=23ms status=404 bytes=840
2016-01-29T18:56:14.320423+00:00 app[web.1]: source=rack-timeout id=76e1c73d-dd22-446f-89b4-096dcbae6f6c wait=0ms timeout=20000ms state=ready
2016-01-29T18:56:14.320609+00:00 app[web.1]: source=rack-timeout id=76e1c73d-dd22-446f-89b4-096dcbae6f6c wait=0ms timeout=20000ms service=1ms state=active
2016-01-29T18:56:14.324730+00:00 app[web.1]: Started GET "/dsfd" for 64.245.0.218 at 2016-01-29 18:56:14 +0000
2016-01-29T18:56:14.331095+00:00 app[web.1]:
2016-01-29T18:56:14.331099+00:00 app[web.1]: ActionController::RoutingError (No route matches [GET] "/fddf"):
2016-01-29T18:56:14.331100+00:00 app[web.1]: vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.4/lib/action_dispa
logs if not found:
Processing by UsersController#show as PNG
2016-01-29T18:49:20.952915+00:00 app[web.1]: Parameters: {"id"=>"thirdlogo"}
2016-01-29T18:49:20.959096+00:00 app[web.1]: User Load (3.1ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT 1 [["id", 1]]
2016-01-29T18:49:20.963937+00:00 app[web.1]: Completed 404 Not Found in 11ms (ActiveRecord: 4.7ms)
2016-01-29T18:49:20.985358+00:00 heroku[router]: at=info method=GET path="/users/thirdlogo.png" host=www.faskyn.com request_id=e913ee09-3837-4da6-8e7f-f3eebc15de00 fwd="64.245.0.218" dyno=web.1 connect=0ms service=30ms status=404 bytes=840