Using BIG DATA implementations onto Software Defined Networking

Using BIG DATA implementations onto Software Defined Networking

• 1.
  Using BIG DATAimplementations onto Software Defined Networking Mohammed Najm abeer Tariq Mohamed Qasim DEPT. of comput.engineering DEPT.of comp.science DEPT.of comp.science Technology Technology University Technology University Technology University Baghdad-Iraq Baghdad-Iraq Baghdad-Iraq mustafamna@Yahoo abeer28003@Yahoo mohammad.qassim2002@gmail Abstract An elastic , effective, activety or intelligent ,graceful networking architecture layout be desired to make processing massive data. next to that ,existent network architectures be considerably incapable for cleatting the huge data. massive data thrusts network exchequers into border it consequence with in network overcrowding ,needy achievement, then permicious employer exprtises. this offered the current state-of-the-art research affronts ,potential solutions into huge data networking notion. More specifically, present the state of networking problems into massive data connected intrequirements,capacity,running , data manipulating also will introduce the architectures of MapReduce , Hadoop paradigm within research requirements, fabric networks and software defined networks which utilizized into making today’s idly growing digital world and compare and contrast into identify relevant drawbacks and solutions. Keywords:Big Data; MapReduce; Hadoop; SDN; 1. INTRODUCTION at present times, implementations be fermantly growing a rate such data be mountng within the world.Legal Studies be displayed for 2020 the world be growing 50 times a sizes of data had into 2011, whom currently 1.8 zetta bytes or 1.8 trillion gigabytes of data [18]. A natural cause into a acute mounting into data making a storage along years merely decline into fees from a store. IT industry making a fees from store become low then implementations be able from provision data exponential rates. That will get an affront from making present network buildings in what way running then execute these massive data to be used to utilizing information [1, 5 - 6, 16]. numerous huge data implementations action or requirements work in a good-time. implementations required into make, storage then operates a big quantities from information whom decrease big accord from quantity ,need for a network. When looking at data from a networking perspective, multi-level regions be required to be reconnoiter which contain network formatting , parallel structures then huge data progressing algorithms, the data recovery then confidentially problemss [13]. The topic massive data as yet afresh rousing parts from research among the Information Technology staff , being demand on extremely regard into the years onto derive applying onto network theory,huge data being described as any aggregate or data-in-motion and many of its applications have real-time requires into making effective especially in an education/research environment [8]. the industry, common implementations utilize to massive data contain for analyzing the data to come up with efficient conclusions in implementation domains like astrophysics, particle physics (e.g., CERN research), biological science (e.g., healthcare), geological science, social networking (e.g., Facebook), trend prediction (e.g., google flu), optimizing route delivery(e.g., UPS package delivery, fuel tracking) [13]. International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, August 2017 139 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 2.
  A typical organizationhas a limited network infrastructure and resources which able be catching that sizes into traffic flows whom cause standard services (e.g., Email, Web browsing, video streaming) being strained. these making come down network execution affecting bandwidth and exposing hardware requirements from tools like a firewall progressing be making over whelmed [13]. This paper presents a comprehensive survey on the network theory of big data. This work startsby introducing the concept of big data and networking theory by giving some background information on the state of networking issues related to capacity, management and data processing in Section II. In Section III briefly presents the architecture of Big Data which includes the MapReduce paradigm and Hadoop distributed architecture, fabric network infrastructure and software defined networks (SDN). We present architectures, designs techniques using theorized solutions to handle today’s idly growing digital world and compareand contrast them to identify relevant problems and solutions. Finally, we discuss, evaluate, and state conclusions on the analysis of the defined Big Data networking concepts in Section IV. 2. BIG DATA NEEDS this introduces making network, a different data types, data flow needs and implementations from massive Data. 2.1. meet the requirements for the massive Data Network Huge data requires a good execution request from a networks buildings implementations that network needs being to extra efficient, flexible then have some form into implementations knowledgs. Huge data network architecture making be deals with a distributed architecture so as to planning , making accessibility from distributed resources when together making in parallel on a single job [7]. Massive data implementations needs starts a progressing big sizes from the information makes a bigger as data propagated towards network. Table 1 shows the comparison of traditional and massive data kinds 11]. 2.2.Needs from different data kind Features into massie data kinds being so diverse of the classic data patterns. Huge data implementations which needs a good -time realization needs the big data sets being defective of a little growing volums onto progressing . The system can not able to executed onto perfect online transaction processing , while utilizing the common SQL analysis models [11]. Massive data requires a further agile flat horizontally scaled data base environment being be able to catching differents unstructured datakinds within complicated , unknown data relationships amongst each other in distributed network architecture. a little older data bases modules making efforted to making a mutate onto building a NewSQL, unless generality have abandoned SQL for new models such as NoSQL [8]. These new databases making difficult or International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 140 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 3.
  hard to usehuge data’s analytical devices ,responsibilities. Figure 1 shows the throughput comparison of RDMS and No SQL data base over the volume of data. 2.3. Network Flow needs network infrastructure a long the years has been adapted in a 3-tier architecture contains computing, storage, networking. the trends of network traffic is used to as “north-south” which means it usually starts at the end user then goes down the integrated stack to the server than to the database. With the needed of massive data in networks, the traffic pattern requires to change. Massive data structure depending on a horizontal scale ,utilizes a distributed approach among the nodes, the traffic demands between the server and storage nodes are much higher than the typical data flow between servers and end users. Data is no longer just being created by external sources, generated from a number of devices and applications. This type of traffic flow is referred to as machine-to-machine/virtual machine network traffic or “east-west” and it needs to be supported when creating a high-execution scalable huge data network. 3. BIG DATA ARCHITECTURE we present architecture and research challenges of MapReduce and Hadoop [2,3,4]that are used to huge data manipulating. introducint a fabric network technology [19] utilized into massive data infrastructure over within characteristics ,requirements. briefly introduced software defined networks (SDN), that more popular onto huge data execution. 3.1. Hadoop and MapReduce MapReduce a core component of the Hadoop Apache software framework , a kind from a programming module whicht used to implemented into different onto languages (e.g., Java, C++) which utilizing to progressing massive data [17]. these kinde from the software device be applications into smallerfragments or blocks which are then sent out to nodes in a cluster or map. It uses a map function that will able to filter, sort, and distribute jobs to various nodes and also uses a reduce function to collect the results from those jobs so they can resolved into a single value to be used for efficientanalysis (Figure 2 ) The MapReduce consists of a job tracker, task trackers, and sometimes a job history server [23].The job International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 141 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 4.
  tracker is usedas the master node that is in charge of managing resources and jobs. The task tracker is used to be deployed to each node in order to run the map and help with some of thecluster task load. The job history server is used to track finished jobs and can be deployed as an independent function. MapReduce operates in parallel across vast cluster sizes, while jobs can bedivided across many different servers [17]. MapReduce has fault-tolerance where each nodesends status updates to the master node, who can re-assign jobs to functioning nodes in cases of node failure. On the other hand, Hadoop is a management software framework that plays an important role inbig data analytics. Hadoop is capable of cataloging, managing, distributing, and queryingunstructured large data sets rapidly across many nodes within a distributed network environment.It uses a Hadoop distributed file storage system (HDFS) for storage that divides data into blockswhich is distributed and stored on multiple nodes. In order to process the data, Hadoop usesMapReduce to break down data for the nodes to process and sort in parallel (Figure 3). The mapprocedure is responsible for filtering and sorting and the reduce procedure focuses on summarytasks. It supports high speed transfer rates and is capable of resilient uninterrupted operation insituations when there is node failure [20]. The infrastructure divides up the nodes into groups orracks. Hadoop is an excellent framework for applications using large search engines such asGoogle Together Hadoop and MapReduce provide the current popular choice for implementation of bigdata infrastructure [10]. A typical Hadoop network structure consists of a slave (data node, tasktracker), master (name node, job tracker) and a client [9]. The client is basically the user interfaceor query engine. The data nodes are used as storage for the data that contain smaller databasessystems and are horizontally distributed across the network. The task tracker is used to process the broken down fragments of the task that has been distributed to a node. The name node maintains a location index of all the other nodes found in the network, so it knows where the specific data is located in which data node. The job tracker is the software job tracking mechanism that is used to transfer and aggregate request search queries (tasks) through to the tasktracker nodes so the end user can perform information analysis on the result. International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 142 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 5.
  3.2. Making aNetwork substructue onto massive Data Atopology depending on a concatenation of switches Known as a leafs which take the access layer and these switches be moer meshed onto spine switches directly building a “fabric” network (Figures7 - 8). Every access-layer switch be one hop away onto each other, basically decreasng a likelihood of bottlenecks then reducing the time latency [22]. A leaf-spine topology can function in layer 2 and 3 that leads switching, routing between the leaf and spine layer. Fabric is said into grown connectivity ,flexibility in networks which executet dynamic virtual environments , prepared for a converged scalability ofdevices. The fabric network topology is defined as “when network nodes are connected into each other using one or many network switches” [19] . International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 143 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 6.
  Fabric networks arebased on spreading the flow of network traffic across many physical links,which outputs an increase in higher throughput, bandwidth and performance over typicalbroadcast networks [15]. A flatter converged network is simpler and cheaper to install, while growing traffic efficiency ,decreasing congestion. Fabric networks have the benefit upon a tree topology because they use layer 2 network paths which help with load balancing and reducingredundancy [21]. A fabrics convergence feature allows multiprotocol communication andtransparency extending to all locations in the network under a single environment. This allows theimplementation of consistent services and a greater network intelligence management policy thatcan provide efficient and secure communication [21]. different characteristces into fabric above a structural tree based topology. first into a fabric topology depending on staffing a point-to-point conexion between nodes utiliziing a single hop distance in relation to the switching process, that greatly decreases latency in the network. Second, may be tolerable to perform a switching fabric utilizing virtualization that permit to diverse networking combinations into perform as one then support to enhance switching execution. likely into make a pool of virtual switches which decrease manual work from the executivor.Third, via having a flat architecture of a single extended environment, it is easier supplier extenination into multi uses areas of the network, when makinging a data center fabric topology[15]. Expansion and enhancement done easily by creating virtual domains or by making point-to-point cconexions. 3.3. Software Defined Networking substructure SDN [22, 26] were progressing into a network arrangement to make network administrators runs manipulating planes. The massive Data connections uses to run one to many connections of client to databases to supply services and manage a high end implementation s so as to load massive amounts of dynamic unstructured data that traditional networks hard to cure and treat. meaning time a data-planes tools link and react on a standard patterns. detaching the control plane and data plane (packet forwarding) make changes who time locked system to an open , action it imaginable to disposition network tools working. detachs to 3-layers: application, control and infrastructure/data (Figure 9). International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 144 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 7.
  With a viewto performance employ SDN architecture at supplying programmable network communicate one be fit run these practicability ,employment about a network by software programs. right away running planes grants an originative trend to leverage adjust the control plane to tool up a elastic and adaptable topology which be used to work with multi uses networked applications and services. Open Networking Foundation supplies the mostly gets the SDN: “Software-Defined Networking becomming the network building wherever the control plane separated of forwarding plane , then fair to programmable” [22]. Figure 10 explain a structure about massive data which programming operations , huge data assure, programming , scheduling and triggering. 3.3.1. parcticabilities about SDN goal from Software Defined Network making the SW which has been used to manage the network [27], to layer of abstraction the goal to use an elastic network running system to mode huge Data demands performance and flexible. The Switches at most forwarding tools s controlling routing decisions which gain processed through central controller. A concept from the programmable networks , decoupled data and control plan exposes a rule to whom a SDN controller performance treats. A control plane connects the channel utilized to replacing indexes (messages) amongst logical network tools for forwarding and running . make the most of the available products of SDN set it up with a North Bound plane who employers utilized a programmability by an (API)onto re-attampt the real time statics , service utilizing . data plane also known as the forward plane which all the information set out to switched in a network which traffic evaluted by then measured to service utilizing. Utilizing these network in an application layer for whom custom-made applications utilized. The Virtualized networks [27] utilized for getting an elastic and resillient . then clients beginning thrust accordingly you utilized a service to holder virtual network iunderlying that utilizing the Virtual Private Server (VMs) to build the customized topologies simplifying the dynamic network re-arrangment utilizing the centralized control plane executed within a software. International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 145 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 8.
  3.3.2. Traffic-Aware Architecture ANorthbound API connecting SDN controller utilizes the additional implementation to relocating. Open Flow be used into conform the callbration from a network tool programming , arrangement in the midst controllers , switches. operation Open Flow contains of 2 - rules that identify the fields, running the packet headers then the instructions make to competitions. Through crossing controller forwarding a switch rule then path from a packet. Effectiveness the controller supples established in advance identify the principles which be executed over the static or dynamic SDN control software. the administrator will decide to use a flow to traffic utilizing models, applications , cloud re-resources. During the Open Flow considered a strongly techniques which existence utilized the operations SDN control plane data which has deiciency while taking into consideration a real-world hardware requirements. to be further essential flow a confiqurations which it has suggested by Curtis[28] the authors of Networking to a huge Data moduler Curtis scanning the delineation Devo Flow to “devolve control by cloning rules whenever flow built utilizing the wild cards.” work done in [13] scaling the overhead by Open Flow so that possess further enhancement to flow running to massive Data scanning applications networking utilizies local jobs to switches owns local re-locate decisions does not need to contact a controller. Figure 11 shows SDN controller and flow table connections to Open Flow. International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 146 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 9.
  3.3.3. Application-Aware Architecture Aimof massive data networks being permanently enhancing effectiveness onto a networks during network arrangements , programmability. doing this impotance from network infrastructure be aware of implementation deficiencies capable of service its requires. A technigues onto service implementation requires to utilize switches pre-rendered flow information into control plane so as to locate , arrange scheduled flows. the flows implement-level information and disorder be contain at the transport layer from huge Data implementations. that participate multiple data transfers called as shuffles from mappers and flow tables connecting into specific sorting of flows. enhancement the flows both hardware and application requirements deficiency to orchestrate a correlation for seamless reconfiguration of resources and upper-layer needs. Due to weak implementations by Hadoop in heterogeneous clusters optimizers have been proposed. OF Scheduler runs MapReduce jobs, asses network traffic, and load-balances traffic on the links decreasing the time , takes jobs to finish based on the demand of MapReduce tasks to enhance the execution [29]. Heavy loaded links are off loaded by preference of load balancing flows especially duplicated jobs, larger flows can affect global cost so offloading happens to large flows to maximize rendering showing and prevent the re-scheduling [13]. REFERENCES [1] Han, Jing, et al. “Survey on NoSQL database”. Pervasive computing and applications (ICPCA), 20116th international conference on. IEEE, 2011.. [2] Lam, Chuck. Hadoop in action. Manning Publications Co., 2010. [3] M. R. Jam, L. M. Khanli, M. S. Javan and M. K. Akbari, “A survey on security of Hadoop”, 2014 4thInternational Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, 2014, pp.716-721. [4 Borthakur, Dhruba. “The hadoop distributed file system: Architecture and design”, Hadoop ProjectWebsite 11, No. 21, 2007. [5] X. W. Chen and X. Lin, “Big Data Deep Learning: Challenges and Perspectives,” in IEEE Access,vol. 2, pp. 514-525, 2014. [6] Sagiroglu, Seref, and DuyguSinanc. “Big data: A review.” Collaboration Technologies and Systems(CTS), 2013 International Conference on. IEEE, 2013. [7] Bushong, Michael. “Six considerations for big data networks”, searchsdn.techtarget.com, Accessed Web on March 2016 [8] Kvernvik Tor and Matti Mona. “Applying big-data technologies to network architecture”, EricssonReview, 2012 [9] Guohui Wang, T.S. Eugene Ng, and AneesShaikh, “Programming your network at run-time for bigdata applications”, In Proceedings of the first workshop on Hot topics in software defined networks (HotSDN '12). ACM, New York, USA, pp. 103-108, 2012 [10] Idiro Analytics, “Hadoop”, http://idiro.com/about-idiro/our-technology/ Accessed Web in May 2016 [11] Introduction to big data: infrastructure and networking considerations, Juniper Networks White Paper2012. [12] Jain, Raj. “Networking issues for big data”, Class Lecture, Washington University in St. Louis 2013 [13] Lin Xiaodong, MisicJelena, ShenXuemin, and Yu Shui, “Networking for big data”, Boca Raton:CRC Press, 2015 Print. International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 147 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
• 10.
  [14] Bertolucci, Jeff.“Big data development challenges: talent, cost, time”. Informationweek.com. Web.Aug 18, 2012. [15] Borovick, Lucinda and Villars and L. Richard, “The critical role of the network in big dataapplications”. Cisco White Paper, April, 2012. [16] Chen, Min, Shiwen Mao, and Yunhao Liu. "Big data: a survey." Mobile Networks and Applications19.2 (2014): 171-209. [17] Dean, Jeffrey, and Sanjay Ghemawat. "MapReduce: simplified data processing on large clusters."Communications of the ACM 51.1 (2008): 107-113. [18] Mearin, Lucas. “World’s data will grow by 50x in next decade, IDS study predicts”. Computerworld.com, Web Accessed on May 2016. [19] Eneboe, Michael K., and Andrew D. Hospodor. “Isochronous switched fabric network”, U.S. Patent [20] Jeffrey Dean and Sanjay Ghemawat, “MapReduce: simplified data processing on large clusters”, TheACM Communication Magazine, Vol. 51, Issue. 1, pp. 107-113, January 2008 [21] Borovick Lucinda and Villars L. Richard, “The critical role of the network in big data applications”.Cisco White Paper, April 2012. [22] Banks, Ethan. “Data center network design moves from tree to leaf”. Search data center.tech target.com, Packet Pushers Interactive, Accessed on June 2016 [23] Jeffrey Dean and Sanjay Ghemawat, “MapReduce: simplified data processing on large clusters”. TheACM Communication Magazine, Volume 51, Issue 1, pp. 107-113, January 2008. [24] IBM, “Bringing big data to th enterprise”,https://www01.ibm.com/software/in/data/bigdata/ Accessed Web on June 2016. [25] McDougall, Richard. “Is your cloud ready for big data?” Strata Conference – Co-presented by O’Really Cloudera, New York, Oct 28- 30, 2013 [26] Open Network Foundation, “Software-defined networking: The new norm for networks,” White Paper, April 2012 [27] Open Networking Foundation, “SDN Architecture Overview”, White Paper V1.0, December 2013. [28] Curtis, A. R., Mogul, J. C., Tourrilhes, J., Yalagandula, P., Sharma, P., and Banerjee, S., “Devoflow:Scaling flow management for high-performance networks”, ACM SIGCOMM Computer Communication Review, 41(4): 254-265, July 2011 [29] Zhao Li, Yao Shen, Bin Yao, MinyiGuo, “OF Scheduler: A Dynamic network Optimizer for MapReduce Heterogenous Cluster”, International Journal of Parallel Programming, Volume43, Issue 3, pp. 472-488, June 2015. International Journal of Computer Science and Information Security (IJCSIS), Vol. 15, No. 8, Augus 2017 148 https://sites.google.com/site/ijcsis/ ISSN 1947-5500