Javascript: How to create Transferable Object for JSON array

Question

I converted one of my existing application to use the "worker" to fetch data from database tables, massage the data to the required format, and to finally send the data to the main-thread for displaying on the UI (User Interface).

The required data format for the UI is an array of JSON objects. [JSON{}, JSON{}, ....]. I am able to send the data from the worker thread to the main thread using the post-message which is using the "structured cloning" copying method. I found that using the "worker" to fetch the data, takes slightly more performance-time than implementing the data-fetching directly in the main-thread itself. I am hoping that by using the "Transferable Objects", I might be able to bring the performance time equal or better than implementing the data fetching in the main-thread.

I tried to use the "Transferable Objects" for transfering the data from the worker to the main thread. But I am not successful since only ArrayBuffer is transferable. The data I fetch from the database, and processing in the worker for sending to the main-thread, are mainly text or string data. Each database row that is fetched is an appointment which is converted (after massaging) into a JSON object as displayed in the code below. I have nearly a thousand appointments to process. When I send the data as single JSON{} to the main thread, it takes considerably more time. Sending the [JSON{}] as an array to the main-thread is much faster. Hence I am looking for ways to send the array of [JSON{}] from the worker to the main-thread and then to retrieve the array in the main-thread for passing to the UI in-turn.

Could anyone please throw some light on how to use the Transferable object in this case?

    let gl_calendarEventArray = [];
    
    //In a loop the array is filled with JSON objects
    
    gl_calendarEventArray.push({
            id: appointment.appointment_id,
            clientIssueFlag: appointment.clientIssueFlag,
            apptIssueFlag: appointment.apptIssueFlag,
            issueTypeFlag: appointment.issueTypeFlag,
            recur_appt_id: appointment.recurring_appointment_id,
            appt_type_id: appointment.appointment_type_id,
            branch_id: appointment.branch_id,
            resourceId: appointment.tutor_id,
            student_branch_id:appointment.student_branch_id,
            student_id: appointment.student_id,
            title: title,
            tutor_id: appointment.tutor_id,
            tutor: appointment.tutor_name,
            service: appointment.service_name,
            student: appointment.student_name,
            start: appointment.start_date_time,
            demo: appointment.demo,
            end: appointment.end_date_time,
            status_id: appointment.appointment_status_id,
            duration: duration,
            area_of_focus: area_of_focus,
            student_notes: student_notes,
            appointment_notes: appointment_notes,
            primary_parent_name: primary_parent_name,
            primary_parent_email: primary_parent_email,
            primary_parent_phone: primary_parent_phone,
            secondary_parent_name: secondary_parent_name,
            secondary_parent_email: secondary_parent_email,
            secondary_parent_phone: secondary_parent_phone,
            current_grade_level: current_grade_level,
            school_name: school_name,
            color: color,
            textColor: textColor,
            additional_info:additional_info
        });         

//After the loop is completed, the data is sent to the main-thread
//by a postMessage which is using the structured copy now.

self.postMessage({msgType: 'CalendarEvent', msg: gl_calendarEventArray});

Kaiido · Accepted Answer · 2021-04-10 03:35:32Z

1

Simply put, you don't.

Only transferables can be transferred, and plain JS objects aren't.

If we absolutely wanted to "transfer" some data, we'd have to first serialize these objects in a form it can fit in an ArrayBuffer (because other transferables can't hold arbitrary data), if we're lazy we could use JSON to stringify it and then convert that string to an ArrayBuffer, e.g through a TextEncoder. If we have time and resources and our formats is well suited for it, we could also build our own serializer and parser.

Then we'd transfer that serialized form to the other side, which would still have to parse it back to the JS Object.

So instead of having just two JS objects, we have two JS objects + an ArrayBuffer holding the serialized data... (and two more strings representations if we were lazy and used JSON). That's not really what I'd call an optimization...

The "magic" in transferables is that you can have a memory slot in one thread be directly accessible to the other thread, avoiding memory bloat and useless Garbage Collection. However transferring transferables is not necessarily faster than cloning them, and doing the serialization + parsing yourself will certainly be a lot less performant than the browser's internal structured cloning.

But, if all you are doing in that Worker is really to fetch that data and there is no CPU intensive task, then just do so from the main context, fetching is done in parallel and is not CPU intensive.

answered Apr 10, 2021 at 3:35

Kaiido

139k15 gold badges266 silver badges333 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Matt Over a year ago

Kaiido: Thank you for your response and clarifying that it t is not really worth the effort to try using the ArrayBuffer. In the worker, I fetch data and do some data transformations. But it is not very CPU intensive. My data to be fetched is dependent on the calendar dates. Based on the user's choice, it could be for one day, for one week, or for a month. If the data to be fetched is for a week or month, I found by experiment that by employing two workers in parallel, I could save on the total elapsed time. Hence using the worker(s) is still beneficial except for the case of the "one-day".

Kaiido Over a year ago

Then continue using Workers if you measured it's beneficial, but don't try transferring the data, you wouldn't win anything.

Collectives™ on Stack Overflow

Javascript: How to create Transferable Object for JSON array

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related