5 disadvantages of apache spark that any developer should be aware of?

Spark has many advantages, but it has certain limitations as well, some of them are:-

In apache, the spark developer needs to optimize their code manually since there is no automatic optimisation.

[{"selector":"#anim-7b2c8f77-fe92-46c4-9704-63dcaea414d0 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Spark does not have any file system, and it relies on Hadoop or object storage for storing the data.

[{"selector":"#anim-e3fb441a-d0b2-44b3-a20c-cc8ee74c3ac4 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Similar to Hadoop, spark also has small files issue. Spark performance degrades when it has to process many small files.

[{"selector":"#anim-e8537814-c22c-4f1d-b0ba-a48be42d1481 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Since spark is an in-memory computation engine, it requires massive RAM for the processing ,which adds cost.

[{"selector":"#anim-032adb2d-5f9d-431a-b88d-9b80a91058c4 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Apache spark has high latency as compared to apache spark.

[{"selector":"#anim-8563d26d-04c2-42d3-80fc-e3a5586fa540 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Click to learn more

[{"selector":"#anim-238d3336-30fd-490c-9afd-2b13e35bc514 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Spark has many advantages, but it has certain limitations as well, some of them are:-

In apache, the spark developer needs to optimize their code manually since there is no automatic optimisation.

No automatic optimization

Spark does not have any file system, and it relies on Hadoop or object storage for storing the data.

No file system

Similar to Hadoop, spark also has small files issue. Spark performance degrades when it has to process many small files.

Small file issue

Since spark is an in-memory computation engine, it requires massive RAM for the processing ,which adds cost.

Expensive

Apache spark has high latency as compared to apache spark.

Latency

Click to learn more