BigData / Hadoop basics
Explain HDFS Read a file workflow.
Client opens the file it wishes to read by calling open() on the Distributed FileSystem (HDFS).
DistributedFileSystem makes an RPC call to the name node to determine the locations of the blocks for the first few blocks in the file.
For each block, the name node returns the addresses of the data nodes that have a copy of that block and data nodes are sorted according to their proximity to the client.
DistributedFileSystem returns an FSDataInputStream to the client for it to read data from. FSDataInputStream in turns wraps the DFSInputStream which manages the data node and name node I/O.
Client calls read() on the stream. DFSInputStream which has stored the data node addresses then connects to the closest data node for the first block in the file.
Data is streamed from the data node back to the client, which calls read() repeatedly on the stream. When the end of the block is reached.
DFSInputStream will close the connection to the data node and then finds the best data node for the next block.
Dogecoin! Earn free bitcoins up to $250 now by signing up.
Earn bitcoins upto $250 (free), invest in other Cryptocurrencies when you signup with blockfi. Use the referral link: Signup now and earn!
Using BlockFi, don't just buy crypto - start earning on it. Open an interest account with up to 8.6% APY, trade currencies, or borrow money without selling your assets.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
More Related questions...