Bleeding Edge Version of BackgrounDRb for better memory usage

Update: Do not forget to get rid of autogenerated backgroundrb etcetera files that were generated during last run of rake backgroundrb:setup.

A plain fork() is evil under Ruby and hence there are some issues with memory usage of BackgrounDRb. I have made some changes to packet library so as it uses fork and exec rather than just fork for better memory usage. However, there were quite bit of changes and BackgrounDRb is affected by them. You can try bleeding edge version as following.Also you need rspec for building packet gem.

clone the packet git repo:

git clone git://github.com/gnufied/packet.git
cd packet;rake gem
cd pkg; sudo gem install --local packet-0.1.6.gem

Go to your vendor/plugins directory of your rails directory and remove or
backup older version of backgroundrb plugin and backup related config
file as well.

from vendor/plugins directory:

git clone git://github.com/gnufied/backgroundrb.git
cd RAILS_ROOT
rake backgroundrb:setup
./script/backgroundrb start

Let me know,how it goes.

18 thoughts on “Bleeding Edge Version of BackgrounDRb for better memory usage

  1. Melvin Ram

    Hey Hemant,

    I’m running this version but I have problems when I try to get data using a model that uses a acts_as_list plugin. Do I need to add something to my code to get this to work?

    ~ mel

    Reply
  2. Pingback: A Place for my head » Unthreaded threads of hobbiton

  3. Walther

    Hi Hemant,

    this is a great plugin – I sure would give my 1 if you’d consider pushing for including it with Rails 2.2!!

    Anywas, I’m not sure if the above patch to packet applies to me – I’m on packet 0.1.5, and my backgrounDRB Changelog has last entry 2008-02-28

    My problem is that I can do

    invoice = Invoice.find(26)
    invoice.print_invoice “some_path_and_filename”

    from the script/console just perfectly

    But when I try calling invoice.print_invoice from my billing_worker.rb – it gives me the first logger.info – and that is it. I do not know if I can debug it any further – and I do not know if there is some Rails specific environment, I’m missing!

    I’ve attached my billing_worker if you can see me doing something terribly wrong (I know it’s not really sleak and elegant code) – can you spot it will not fly?

    cheers,
    walt

    class BillingWorker < BackgrounDRb::MetaWorker
    set_worker_name :billing_worker
    def create(args = nil)
    # this method is called, when worker is loaded for the first time
    logger.info “Faktura baggrundsjob startet!”
    end

    def build_invoice(invoice_id = nil)
    invoice = Invoice.find(invoice_id) unless invoice_id.nil?
    logger.info “Faktura ID #{invoice_id} skal dannes!” # I only reach this!

    unless invoice.nil?
    result = false
    logger.info “Faktura ID #{invoice_id} dannes!”
    result = invoice.print_invoice “public/forms/invoices/html/#{invoice.invoice_number}.html”
    #result = %x[script/get_invoice #{invoice.id} #{invoice.invoice_number}]
    if result
    logger.info “Faktura nr #{invoice.invoice_number} blev dannet – som HTML!”
    result = %x[script/get_invoice_pdf #{invoice.invoice_number} #{invoice.invoice_number}]
    #result=false
    if result
    logger.info “Faktura nr #{invoice.invoice_number} blev dannet – som PDF!”
    invoice.pdf_resource = “forms/invoices/pdf/#{invoice.invoice_number}.pdf”
    invoice.invoice_items.each { |ii| Task.find(ii.task_id).update_attribute( :invoiced_at, DateTime.now) unless ii.task_id.nil? }
    invoice.save
    logger.info “Faktura nr #{invoice.invoice_number} blev dannet med faktura linier!”
    else
    logger.info “Faktura med ID #{invoice_id} blev ikke dannet – som PDF!”
    end
    else
    logger.info “Faktura med ID #{invoice_id} blev ikke dannet – som HTML!”
    end
    else
    logger.info “Faktura med ID #{invoice_id} blev ikke fundet!”
    end
    end

    end

    Reply
  4. Walther

    Hi Hemant,

    it’s me (again) > worker = MiddleMan.worker(:billing_worker)
    => #
    >> worker.build_invoice(27)
    {:type=>:do_work, :worker_method=>:build_invoice, :worker=>:billing_worker, :data=>27}
    => nil
    >>

    I get this:

    # script/backgroundrb
    true

    /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:478:in `const_missing’: uninitialized constant Invoice::ApplicationController (NameError)
    from /Users/walther/Documents/RailsProjects/froeslev/app/models/invoice.rb:98:in `render_string’
    from /Users/walther/Documents/RailsProjects/froeslev/app/models/invoice.rb:62:in `render_to_string’
    from /Users/walther/Documents/RailsProjects/froeslev/app/models/invoice.rb:47:in `print_invoice’
    from /Users/walther/Documents/RailsProjects/froeslev/lib/workers/billing_worker.rb:15:in `build_invoice’
    from /Users/walther/Documents/RailsProjects/froeslev/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:235:in `send’
    from /Users/walther/Documents/RailsProjects/froeslev/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:235:in `process_request’
    from /Users/walther/Documents/RailsProjects/froeslev/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:219:in `receive_data’
    from /Library/Ruby/Gems/1.8/gems/packet-0.1.5/lib/packet/packet_worker.rb:53:in `receive_internal_data’
    … 15 levels…
    from /Library/Ruby/Gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:20:in `run’
    from /Users/walther/Documents/RailsProjects/froeslev/vendor/plugins/backgroundrb/server/lib/master_worker.rb:166:in `initialize’
    from script/backgroundrb:60:in `new’
    from script/backgroundrb:60

    Reply
  5. Walther

    Hi Hemant,

    sorry to post wall-to-wall comments – false
    File.open(path_and_file, “w “) do |f|
    f.puts( htmlstring)
    f.close
    end
    true
    # rescue
    # false
    end

    Reply
  6. Walther

    Okay,

    so I try just once more 🙁

    I wanted to show you that when I call my invoice.print_invoice from the console – everything is ok – but when I call it from my billing_worker – I only get the logger to write the first logger.info entry, and it does not finish work at all.

    the first comment holds my billing_worker, the next comment shows how I call the worker from the console – and the stack trace from backgrounDRB started without the start argument. the comment right above shows the print_invoice method in my invoice.rb model

    can you help me?

    cheers,
    walt

    Reply
  7. Hemant Post author

    Walther,

    It would be a good idea to contact mailing list for such questions. But anyways, I guess, looks like you need to require some controller that you are using in your model or some crap like that. WordPress is not letting code pass cleanly and hence you can use pastie to paste the code.

    Reply
  8. Melvin Ram

    I agree there should be a mailing list for asking questions and answering them as I didn’t know there was an update to my question till today.

    I vote for using Google Groups.

    PS: Thanks Hemant for the update. I give it a go over the next week.

    Reply
  9. Lukas Rieder

    Really Important is to REMOVE THE OLD FILES GENERATED FROM backgroundrb:setup!
    This saved my day (and night probably too).

    Thanks for sharing,

    Have a nice day!

    Lukas

    Reply
  10. Pingback: chirp.syxyz.net » Blog Archive » upgrading backgroundrb, the fun way.

  11. Jesse

    i’m trying backgroundrb 1.1, i can start backgroundrb service, but MiddleMan.all_worker_info is empty.

    i’m using packet (0.1.14),chronic (0.2.3)

    i have searched from google, and got many same error about : Invalid worker with name notice_worker and key
    /var/lib/gems/1.8/gems/packet-0.1.14/lib/packet/packet_connection.rb:52:in `ask_worker’

    I’m trying to get packet-0.1.6.gem, it’s very had to reach github for me, cause i’m behind GFW, GFW is suck!

    Reply
  12. Pingback: links for 2008-10-24 « Caiwangqin’s delicious bog

Leave a Reply

Your email address will not be published. Required fields are marked *