Mastering Data Science with Jupyter, Pandas, NumPy, and
School
University of the City of Muntinlupa (Pamantasan ng Lungsod ng Muntinlupa)**We aren't endorsed by this school
Course
CITCS N/A
Subject
Electrical Engineering
Date
Dec 12, 2024
Pages
19
Uploaded by BaronDragon8052
REVIEWERPROF ELEC 2-TheJupyter Notebookis an incredibly powerful tool for interactively developing and presenting data science projects.-A notebook integrates code and its output into a single document that combines visualizations, narrative text, mathematical equations, and other rich media.What is Pandas?Pandasis a Python library used for working with data sets.It has functions for analyzing, cleaning, exploring, and manipulating data.The name "Pandas" has a reference to both "Panel Data", and "Python Data Analysis" and was created by Wes McKinney in 2008.Import PandasOnce Pandas is installed, import it in your applications by adding the importkeyword:(import pandas)Now Pandas is imported and ready to use.Pandas as pdPandasis usually imported under the pd alias.import pandas as pdNow the Pandas package can be referred to as pd instead of pandas.Why Use Pandas?-Pandas allows us to analyze big data and make conclusions based on statistical theories.-Pandas can clean messy data sets, and make them readable and relevant.-Relevant data is very important in data science.What Can Pandas Do?
Pandas gives you answers about the data. Like:Is there a correlation between two or more columns?What is average value?Max value?Min value?Pandas are also able to delete rows that are not relevant, or contains wrong values, like empty or NULL values. This is called cleaning the data.What is NumPy?-NumPy is a Python library used for working with arrays.-It also has functions for working in domain of linear algebra, fourier transform, and matrices.-NumPy was created in 2005 by Travis Oliphant. It is an pen-source project and you can use it freely.-NumPy stands for Numerical Python.Import NumPyOnce NumPy is installed, import it in your applications by adding the import keyword:import numpyNow NumPy is imported and ready to use.NumPy as npNumPyis usually imported under the np alias.import numpy as npNow the NumPy package can be referred to as np instead of numpy.Why Use NumPy?-In Python we have lists that serve the purpose of arrays, but they are slow to process.-NumPy aims to provide an array object that is up to 50x faster than traditional Python lists.-The array object in NumPy is called ndarray, it provides a lot of supporting functions that make working with ndarray very easy.-Arrays are very frequently used in data science, where speed and resources are very important.
What is Matplotlib?-Matplotlib is a low level graph plotting library in python that serves as a visualization utility.-Matplotlib was created by John D. Hunter.-Matplotlib is open source and we can use it freely.-Matplotlib is mostly written in python, a few segments are written in C, Objective-C and Javascript for Platform compatibility.Import Matplotlibimport matplotlibNow Matplotlib is imported and ready to usePyplotMost of the Matplotlib utilities lies under the pyplot submodule, and are usually imported under the plt alias:import matplotlib.pyplot as pltNow the Pyplot package can be referred to as plt.STRING METHODS Creating a stringA string is created by enclosing text in quotes.You can use either single quotes, ', or double quotes, ".A triple-quotecan be used for multi-line strings. Here are someexamples:s = 'Hello't = "Hello"m = """This is a long string that isspread across two lines."""Empty stringThe empty string '' is the string equivalent of the number 0. It is a string with nothing in it. We have seen it before, in the print statement’s optional argument, sep=‘’.
Length To get the length of a string (how many characters it has), use the built-in function len.For example, len('Hello') is 5.Important noteOne very important note about lower, upper, andreplace is that they do not change the original string. If you want tochange a string, s, to all lowercase, it is not enough to just use s.lower().You need to do the following:s = s.lower()Short examplesHere are some examples of string methods inisalphaThe isalpha method is used to tell if a character is a letter ornot. It returns True if the character is a letter and False otherwise.When used with an entire string, it will only return True if every character of the string is a letter.Here is a simple example:
A note about indexIf you try to find the index ofsomething that is not in a string, Python will raise an error. For instance, if s='abc' and you try s.index('z'), youwill get an error. One way around this is to checkfirst,like below:if 'z' in s:location = s.index('z')-capitalize() - converts the first character to upper case-casefold() - converts string into lower case-center() - return centered string-count() - returns the number of times a specified value occurs in a string-encode() - return encoded version of the string-endswith() - returns true if the string ends with the specified value-expandtabs() - sets the tab size of the string-find() - searches the string for a specified value and returns the position of where it was found-format() - format specified values in a string-format_map() - formats specified values in a string-index() - searches the string for a specified value and return the position where it-was found-isalnum() - returns true if all characters in the string are alphanumeric-isalpha() - returns true if all characters in the string are in the alphabet-isdecimal() - returns true if all characters in the string are decimals-isdigit() - returns true if all characters in the string are digits-sidentifier() - returns true if the string is an identifier-islower() - returns true if all characters in the string are lower case-isnumeric() - returns true if all characters in the string are numeric-isprintable() - returns true if all characters in the string are printable-isspace() - returns true if all characters in the string are whitespace
-istitle() - returns true if the string follows the rules of a title-isupper() - returns true if all characters in the string are upper case-join() - joins the elements of an iterable to the end of the string-ijust() - returns a left justified version of the string-lower() - cenverts a string into lower case-istrip() - returns a left trim version of the string-maketrans() - returns a translation table to be used in translations-partition() - returns a tuple where the string is parted into three parts-replace() - returns a string where a specified value is replaced with a specified value-rfind() - searches the string for a specified value and returns the last position of where it was found-rjust() - returns a right justified version of the string-rpartition() - returns a tuple where the string is parted into three parts-split() - splits the string at the specified separator and returns a list-rstrip() - returns a right trim version of the string-splitlines() - splits the string at line breaks and returns a list-startswith() - returns true if the string starts with the specified value-strip() - returns a trimmed version of the string-swapcase() - swaps cases, lower case becomes upper case and vice versa-title() - converts the first character of each word to upper case-translate() - returns a translated string-upper() - converts a string into upper case-zfill() - fills the string with a specified number of 0 values at the beginningREADING AND WRIING FILESOpening a File-Before we can read the contents of the file we must tell Pythonwhich file we are going to work with and what we will be doingwith the file-This is done with the open() function-open() returns a “file handle” - a variable used to perform operations on the file-Kind of like “File -> Open” in a Word ProcessorOpening a File-handle = open(filename, mode)returns a handle use to manipulate the file fhand = open(‘plmun.txt', 'r')-filename is a string-mode is optional and should be 'r' if we are planning reading the file and 'w' if we are going to write to the file.
NETWORKING AND COMMUNICATIONDHCP stands for Dynamic Host Configuration Protocol. It is the critical feature on which the users of an enterprise network communicate. DHCP helps enterprises to smoothly manage the allocation ofIP addressesto the end-user clients’ devices such as desktops, laptops, cellphones, etc. is an application layer protocol that is used to provide:Subnet Mask (Option 1 - e.g., 255.255.255.0)Router Address (Option 3 - e.g., 192.168.1.1)DNS Address (Option 6 - e.g., 8.8.8.8)Vendor Class Identifier (Option 43 - e.g., 'unifi' = 192.168.1.9 ##where unifi = controller)Why Use DHCP?DHCP helps in managing the entire process automatically and centrally. DHCPhelps in maintaining a unique IP Address for a host using the server. DHCP servers
maintain information on TCP/IP configuration and provide configuration of addressto DHCP-enabled clients in the form of a lease offer.Components of DHCPThe main components of DHCP include:DHCP Server:DHCP Server is basically a server that holds IP Addressesand other information related to configuration.DHCP Client:It is basically a device that receives configurationinformation from the server. It can be a mobile, laptop, computer, or anyother electronic device that requires a connection.DHCP Relay:DHCP relays basically work as a communication channelbetween DHCP Client and Server.IP Address Pool:It is the pool or container of IP Addresses possessedby the DHCP Server. It has a range of addresses that can be allocated todevices.Subnets:Subnets are smaller portions of the IP network partitioned tokeep networks under control.Lease:It is simply the time that how long the information received fromthe server is valid, in case of expiration of the lease, the tenant musthave to re-assign the lease.DNS Servers:DHCP servers can also provide DNS (Domain NameSystem) server information to DHCP clients, allowing them to resolvedomain names to IP addresses.Default Gateway:DHCP servers can also provide information about thedefault gateway, which is the device that packets are sent to when thedestination is outside the local network.Options:DHCP servers can provide additional configuration options toclients, such as the subnet mask, domain name, and time serverinformation.Renewal:DHCP clients can request to renew their lease before itexpires to ensure that they continue to have a valid IP address andconfiguration information.Failover:DHCP servers can be configured for failover, where twoservers work together to provide redundancy and ensure that clients canalways obtain an IP address and configuration information, even if oneserver goes down.Dynamic Updates:DHCP servers can also be configured to dynamicallyupdate DNS records with the IP address of DHCP clients, allowing foreasier management of network resources.Audit Logging:DHCP servers can keep audit logs of all DHCPtransactions, providing administrators with visibility into which devicesare using which IP addresses and when leases are being assigned orrenewed.Format1.Hardware length:
This is an 8-bit field defining the length of the physical address in bytes. e.g forEthernet the value is 6.2.Hop count:This is an 8-bit field defining the maximum number of hops the packet can travel.3.Transaction ID:This is a 4-byte field carrying an integer. The transcation identification is set bythe client and is used to match a reply with the request. The server returns thesame value in its reply.4.Number of seconds:This is a 16-bit field that indicates the number of seconds elapsed since the timethe client started to boot.5.Flag:This is a 16-bit field in which only the leftmost bit is used and the rest of the bitshould be set to os.A leftmost bit specifies a forced broadcast reply from the server. If the reply wereto be unicast to the client, the destination. IP address of the IP packet is theaddress assigned to the client.6.Client IP address:This is a 4-byte field that contains the client IP address . If the client does not havethis information this field has a value of 0.7.Your IP address:This is a 4-byte field that contains the client IP address. It is filled by the server atthe request of the client.8.Server IP address:This is a 4-byte field containing the server IP address. It is filled by the server in areply message.9.Gateway IP address:This is a 4-byte field containing the IP address of a routers. IT is filled by the serverin a reply message.10.Client hardware address:This is the physical address of the client .Although the server can retrieve thisaddress from the frame sent by the client it is more efficient if the address issupplied explicity by the client in the request message.11.Server name:This is a 64-byte field that is optionally filled by the server in a reply packet. Itcontains a null-terminated string consisting of the domain name of the server. Ifthe server does not want to fill this filed with data, the server must fill it with all0s.
12.Boot filename:This is a 128-byte field that can be optionally filled by the server in a reply packet.It contains a null- terminated string consisting of the full pathname of the boot file.The client can use this path to retrieve other booting information. If the serverdoes not want to fill this field with data, the server must fill it with all 0s.13.Options:This is a 64-byte field with a dual purpose. IT can carry either additionalinformation or some specific vendor information. The field is used only in a replymessage. The server uses a number, called a magic cookie, in the format of an IPaddress with the value of 99.130.83.99. When the client finishes reading themessage, it looks for this magic cookie. If present the next 60 bytes are options.Working of DHCPThe working of DHCP is as follows:DHCP works on the Application layer of the TCP/IP Protocol. The main task of DHCPis to dynamically assigns IP Addresses to the Clients and allocate information onTCP/IP configuration to Clients. For more, you can refer to the ArticleWorking ofDHCP.The DHCPport numberfor the server is 67 and for the client is 68. It is a client-server protocol that uses UDP services. An IP address is assigned from a pool ofaddresses. In DHCP, the client and the server exchange mainly 4 DHCP messagesin order to make a connection, also called the DORA process, but there are 8 DHCPmessages in the process.The 8 DHCP Messages:1. DHCP discover message:This is the first message generated in thecommunication process between the server and the client. This message isgenerated by the Client host in order to discover if there is any DHCPserver/servers are present in a network or not. This message is broadcasted
to all devices present in a network to find the DHCP server. This message is342 or 576 bytes longAs shown in the figure, the source MAC address (client PC) is 08002B2EAF2A, thedestination MAC address(server) is FFFFFFFFFFFF, the source IP address is0.0.0.0(because the PC has had no IP address till now) and the destination IPaddress is 255.255.255.255 (IP address used for broadcasting). As they discovermessage is broadcast to find out the DHCP server or servers in the networktherefore broadcast IP address and MAC address is used.2. DHCP offers a message:The server will respond to the host in thismessage specifying the unleased IP address and other TCP configurationinformation. This message is broadcasted by the server. The size of themessage is 342 bytes. If there is more than one DHCP server present in thenetwork then the client host will accept the first DHCP OFFER message itreceives. Also, a server ID is specified in the packet in order to identify theserver.Now, for the offer message, the source IP address is 172.16.32.12 (server’sIP address in the example), the destination IP address is 255.255.255.255(broadcast IP address), the source MAC address is 00AA00123456, thedestination MAC address is FFFFFFFFFFFF. Here, the offer message isbroadcast by the DHCP server therefore destination IP address is thebroadcast IP address and destination MAC address is FFFFFFFFFFFF and thesource IP address is the server IP address and the MAC address is the serverMAC address.Also, the server has provided the offered IP address 192.16.32.51 and a lease time of 72 hours(after this time the entry of the host will be erased from the server automatically). Also, the client identifier is the PC MAC address (08002B2EAF2A) for all the messages.3. DHCP request message:When a client receives an offer message, itresponds by broadcasting a DHCP request message. The client will producea gratuitous ARP in order to find if there is any other host present in thenetwork with the same IP address. If there is no reply from another host,then there is no host with the same TCP configuration in the network and
the message is broadcasted to the server showing the acceptance of the IPaddress. A Client ID is also added to this message.Note –This message is broadcast after the ARP request broadcast by the PC to find out whether any other host is not using that offered IP. If there is no reply, then the client host broadcast the DHCP request message for the server showing the acceptance of the IP address and Other TCP/IP Configuration.4.DHCP acknowledgment message:In response to the request messagereceived, the server will make an entry with a specified client ID and bindthe IP address offered with lease time. Now, the client will have the IPaddress provided by the server.5. DHCP negative acknowledgment message:Whenever a DHCP serverreceives a request for an IP address that is invalid according to the scopes that areconfigured, it sends a DHCP Nak message to the client. Eg-when the server has noIP address unused or the pool is empty, then this message is sent by the server tothe client.6. DHCP decline:If the DHCP client determines the offered configurationparameters are different or invalid, it sends a DHCP decline message to the server.When there is a reply to the gratuitous ARP by any host to the client, the clientsends a DHCP decline message to the server showing the offered IP address isalready in use.7. DHCP release:A DHCP client sends a DHCP release packet to the server torelease the IP address and cancel any remaining lease time.8. DHCP inform:If a client address has obtained an IP address manually then theclient uses DHCP information to obtain other local configuration parameters, suchas domain name. In reply to the DHCP inform message, the DHCP servergenerates a DHCP ack message with a local configuration suitable for the clientwithout allocating a new IP address. This DHCP ack message is unicast to theclient.This document covers the basics of how wireless technology works, and how it isused to createnetworks. Wireless technology is used in many types ofcommunication. We use it for networking because it is cheaper and more flexiblethan running cables. While wireless networks can be just as fast and powerful aswired networks, they do have some drawbacks.Reading and working through Learn Networking Basics before this document willhelp you with some of the concepts used in wireless networks.In addition to some background information, this document covers six basicconcepts:1. Wireless signals- what they are and how signals can differ.2. Wireless devices- the differences and uses for receivers and transmitters.3. Wi-Fi Modes- how networks are made up of clients, access points, or ad-hocdevices.4. Wi-Fi Signals- the unique characteristics of Wi-Fi, and how signals areorganized.5. Power and Receiver sensitivity- how far each wireless device can go, andhow well a
router can listen and filter out interference and noise.6. Antennas- how the type of antenna changes the way the router broadcasts.WHAT IS A WIRELESS SIGNAL?Wireless signals are important because they can transfer information -- audio,video, our voices,data -- without the use of wires, and that makes them veryuseful.Wireless signals are electromagnetic waves travelling through the air. These areformed when electric energy travels through a piece of metal -- for example a wireor antenna – and waves are formed around that piece of metal. These waves cantravel some distance depending on the strength of that energy.TYPES OD WIRELESS SIGNALS There are many, many types of wireless technologies. You may be familiar withAM and FM radio, Television, Cellular phones, Wi-Fi, Satellite signals such as GPSand television, two-way radio, and Bluetooth. These are some of the most commonsignals, but what makes them different?FrequencyFirst of all, wireless signals occupy a spectrum, or wide range, of frequencies: therate at which a signal vibrates. If the signal vibrates very slowly, it has a lowfrequency. If the signal vibrates very quickly, it has a high frequency. Frequency ismeasured in Hertz, which is the count of how quickly a signal changes everysecond. As an example, FM radio signals vibrate around 100 million times everysecond! Since communications signals are often very high in frequency, weabbreviate the measurements for the frequencies - millions of vibrations a secondis Megahertz (MHz), and billions of vibrations a second is Gigahertz (GHz). Onethousand Megahertz is one Gigahertz.1. The frequencies from left to right: 2. AM Radio: Around 10MHz3. FM Radio: Around 100MHz4. Television: Many frequencies from 470MHz to 800MHz, and others.5. Cellular phones: 850MHz, 1900MHz, and others6. Wi-Fi: 2.4GHz7. Satellite: 3.5GHz8. Wi-Fi: 5GHzWi-Fi SignalsWhen building a network, you will be using Wi-Fi technology, which has some uniquecharacteristics you will need to know.There are two types of Wi-Fi signal, based on the frequencies they use:
1. 2.4GHz - A lower frequency, this is the more common Wi-Fi technology in use today.Many devices use it, so the signals can become more crowded and interfere with eachother. It can pass through walls and windows fairly well.2. 5GHz - This higher frequency technology is used by fewer devices, and can sometimesachieve higher speeds because the frequencies are less crowded. It cannot pass throughwalls and windows as well as the 2.4GHz band signals, so the range of 5GHz technologyis often shorter.These two types of Wi-Fi are called theFrequency Bands, or just Bands for short.PROJECT MANAGEMENTNetwork Diagram: A graphical representation of a project's activities and their interdependencies, used to planand monitor project progress.Project Management Plan: A formal document that defines how a project is executed, monitored, andcontrolled. It includes all planning documents and baselines.Active Listening: Fully concentrating, understanding, responding, and remembering what is being said incommunication.Baseline: The original plan plus all approved changes. It is used to measure how actual performance deviatesfrom the plan.Brainstorming: A technique used to generate a large number of ideas for the solution to a problem.Cause-and-Effect Diagram: Also known as a fishbone or Ishikawa diagram, it is used to identify potential causesof problems.Contingency Plans: Alternative plans that can be implemented if identified risks become reality.Contract: A binding agreement between two or more parties that is enforceable by law.Controlling Communications: Ensuring that information needs of project stakeholders are met and thatcommunication is effective and efficient.Costs Budget:The allocated funds for a project, broken down into individual cost components.Crashing: A technique to shorten the schedule duration for the least incremental cost by adding resources.Extrinsic Motivation: Behavior driven by external rewards such as money, fame, grades, or praise.Forecasting: The process of predicting future project performance based on current data and trends.Gantt Charts: A type of bar chart that represents a project schedule, showing the start and finish dates ofelementsHistorical Information: Data from past projects that can be used for planning and managing future projects.Identifying Risks: The process of determining risks that could affect the project and documenting theircharacteristics.Initiating: The process group that involves activities necessary to define a new project or a new phase of anexisting project.Initiation: The formal start of a new project or phase, often marked by the development of the project charter.Interactive Communication: Two-way information exchange involving direct interaction between parties.Intrinsic Motivation: Behavior driven by internal rewards, such as personal satisfaction or enjoyment.Management Reserves: Budget within the cost baseline allocated for unforeseen risks.Manager: A person responsible for controlling or administering all or part of a project or organization.Managing Communications: Ensuring timely and appropriate generation, collection, distribution, storage, anddisposal of project information.Oversees Project Task: Supervising and ensuring the completion of specific project activities.
Performing Qualitative Risk Analysis: The process of prioritizing risks for further analysis or action by assessingtheir probability and impact.Planning Communications: Determining the information and communication needs of the project stakeholders.Planning Risk Management: Defining how to conduct risk management activities for a project.Planning Risk Responses: Developing options and actions to enhance opportunities and reduce threats toproject objectives.Power: The capacity or ability to influence the behavior of others within the project.Procurement: The process of obtaining goods and services from external sources.Project:A temporary endeavor undertaken to create a unique product, service, or result.Project Human Resource Management: The processes required to make the most effective use of the peopleinvolved with the project.Project Management:The application of knowledge, skills, tools, and techniques to project activities to meetproject requirements.Project Risk Management: The processes of conducting risk management planning, identification, analysis,response planning, and monitoring and control on a project.Pull Communication: Information is stored and recipients must retrieve it themselves (e.g., intranet sites).Push Communication: Information is sent directly to recipients without them needing to ask for it (e.g., emails).Quality Assurance: The process of ensuring the quality of the project's deliverables and processes.Risk Identification:The process of determining which risks may affect the project and documenting theircharacteristics.Risk Management:The process of identifying, analyzing, and responding to project risks.Risk Response Planning:Developing options and actions to enhance opportunities and reduce threats to projectobjectives.Risk Utility or Risk Tolerance: The degree of variability in outcomes that a stakeholder is willing to accept.Scatter Diagram: A graphical representation used to identify the relationship between two variables.SharePoint Portal: A web-based platform used for document management and storage system, collaboration,and other functionalities.Status Reports: Regular updates on project progress, performance, and any issues encountered.Sunk Cost: Costs that have already been incurred and cannot be recovered.TQM (Total Quality Management):A management approach to long-term success through customersatisfaction, focusing on continuous improvement.Work-Breakdown Structure (WBS): A hierarchical decomposition of the total scope of work to accomplishproject objectives and create deliverables.FOREIGN LANAGUAGELESSON 3
DEPARMENT STORE - DEPAATO