16TB Datawarehouse Appliance using a Sun E20K and Oracle (circa 2008)

We created a 16 TB Data Warehouse Appliance with Oracle and a Sun E20K server and Sun StorageTek 6540 Arrays, circa 2008 pre Oracle Exadata, we wanted to get 6 GB/sec from the Appliance. In that Data warehouse Appliance we wanted to achieve 1 GB/s from each of the arrays. Each Array was connected to 1 dual ported HBA, where each HBA port delivered 4Gbps ie. 2 x 512 MB/s

We tested it by connecting the first storage array, and made sure that it delivered 1 GB/s. Next, we connect the second storage array and made sure we got 2 GB/s. We continued this till we got 6 GB/s when all 6 storage Arrays were connected to the E20K via 6 dual ported HBAs.

I've put the details of how we configured each of the Storage Arrays to deliver 1 GB/s, and the issues that we hit while trying to scale, under the 'Storage' paragraph.

CPU
As our requirement was a sustained throughput of 6 GB/s, we planned on 36 CPU's of the E20K server where each CPU could deliver 200 MB/sec

The rough estimate formula for the number of  CPUs from Oracle is:

<number of CPUs> = <maximum throughput in MB/s> / 200

Memory
We can derive the amount of memory that we need from the amount of CPU's that we are using

<amount of memory in GB> = 2 * <number of CPUs>

Storage:
Calculating the Maximum Throughput of a HBA port:

Lets say we have a really old HBA on our sever, with a 1Gbps port. Then the Maximum Throughput that we can get from that port is 128 MB/s

1 Gbps = 1/8 bits = 0.125 GigaBytes 0.125 x 1024 = 128 MB/sec

Now lets say we replace that HBA with a new one that has 4 Gbps ports. The Maximum Throughput that we can get from that port is 512 MB/s

 4 Gbps = 4/8 = 0.5 GibaBytes
0.5 x 1024 = 512 GB/sec

16 LUNs were created in each Storage Array. Each LUN had 4 disk (3+1 RAID 5), and each disk was 136GB. So we got roughly half a Terabyte (408GB) from each LUN.

To create the above configuration we started with 1 Storage Array, and had it connected to the E20K server. We then measured the throughput we got from a single drive. Next, we created a RAID 5 LUN, to which we kept adding disks. When we noticed that the throughput wasn't increasing that is when we stopped and got our LUN configuration of 3+1 RAID 5.

Issues we hit during the Storage configuration:

• We used vdbench, Oracle's ORION tool and the native "dd" tool to configure the storage. After configuring the LUNs we gave the system and storage to the Oracle team to run their ORION tests. After running some of their tests the Oracle team got back saying that the Storage and I/O sub-system had issues, as it did not deliver the required throughput of 1 GB/s. They said that they double checked it using the dd command.  After getting the hardware back from the Oracle team, we figured the issue actually came from the ORION tool, which back then had a bug(it was single threaded). We also proved to the Oracle team that running a sing "dd" instance did not prove anything. We convinced them by running number of "dd" instances in parallel. We could clearly see that we we increased the number of dd's the throughput started increasing

•  The second issue that we hit was when we tried to scale. After getting 1 GB/s from our first Storage, we connected the second Storage Array, and that gave us 2 GB/s as we had expected. On adding the third Storage Array we saw that the throughput was did not increase, but still delivered 2 GB/s. After analysing the storage sub-system thoroughly we learnt that the issue was with the way we had connected to the Storage Array to the E20K server. The Sun Fire E25K/E20K hot-swap PCI assembly architecture (hsPCI-X/hsPCI+) has two I/O controllers. Each controller provides one 33-MHz peripheral component interconnect (PCI) bus and three 33/66/90 MHz PCI buses for a total of four on each I/O assembly. Therefore, each I/O assembly has four hot-swap component PCI slots. A Sun Fire I/O assembly has a 2.4 Gbyte/sec connection to the rest of the system.So when we connected our first two storage arrays to the I/O assembly we were fine as we were within the 2.4 Gbyte/sec limit, but saw the issue when the third storage array was connected. We resolved the issue by connecting the other arrays to a different I/O assembly (The E20K has 9 I/O assemblies in total).

The above idea and method can be used to create any appliance, particularly a Hadoop appliance.

Processor information in a diagram

In the previous post we have see how to get the processor information on a X86 system running Linux.
http://shettymayur.blogspot.com/2014/03/processor-information-using-dmidecode.html

In this post we will use Portable hardware locality (hwloc) to create is diagram of the processors
on the system that we can use in a whitepaper or a blog entry.

I downloaded hwloc version 1.7.2 from https://www.open-mpi.org/software/hwloc/v1.7/
I user the https://www.open-mpi.org/software/hwloc/v1.7/downloads/hwloc-1.7.2.tar.gz tarball.

Just some information on the OS and Red Hat version that I'm on.

[root@isvx3 ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 5.9 (Tikanga) [root@isvx3 ~]# cat /proc/version Linux version 2.6.18-348.16.1.el5 (mockbuild@x86-012.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-54)) #1 SMP Sat Jul 27 01:05:23 EDT 2013

I created a directory hwloc and then downloaded the tarball into it. Next I untared hwloc-1.7.2.tar

[root@isvx3 hwloc]# pwd /root/hwloc [root@isvx3 hwloc]# ls hwloc-1.7.2  hwloc-1.7.2.tar

Next, I can the 'configure' followed by make and make install.

[root@isvx3 hwloc-1.7.2]# ./configure [root@isvx3 hwloc-1.7.2]# make [root@isvx3 hwloc-1.7.2]# make install

Next I ran 'lstopo' to create the .png and .ps files as follows

[root@isvx3 hwloc-1.7.2]# lstopo isvx3.png [root@isvx3 hwloc-1.7.2]# lstopo isvc3.ps

This created two files isvx3.png and isvx3.ps in that directory

Processor information using dmidecode on Linux

Number of Socket, Cores, and hardware therads are common question that we need to know about a system running an Oracle database, one that is being used as a Hadoop NameNode or a DataNode.
The /proc/cpuinfo gives us fair bit of that information, but the dmidecode gives us some more detail.

We see below that the dmidecode tells us that we have 2 sockets, and that there are 12 hardware threads that are running per socket. Also, we can see that each socket has 6 cores.

[root@isvx3 hwloc-1.7.2]# dmidecode -t processor SMBIOS 2.5 present. Handle 0x0001, DMI type 4, 40 bytes Processor Information         Socket Designation: Node 1 Socket 1         Type: Central Processor         Family: Xeon MP         Manufacturer: Intel(R) Corporation         ID: C2 06 02 00 FF FB EB BF         Signature: Type 0, Family 6, Model 44, Stepping 2         Flags:                 FPU (Floating-point unit on-chip)                 VME (Virtual mode extension)                 DE (Debugging extension)                 PSE (Page size extension)                 TSC (Time stamp counter)                 MSR (Model specific registers)                 PAE (Physical address extension)                 MCE (Machine check exception)                 CX8 (CMPXCHG8 instruction supported)                 APIC (On-chip APIC hardware supported)                 SEP (Fast system call)                 MTRR (Memory type range registers)                 PGE (Page global enable)                 MCA (Machine check architecture)                 CMOV (Conditional move instruction supported)                 PAT (Page attribute table)                 PSE-36 (36-bit page size extension)                 CLFSH (CLFLUSH instruction supported)                 DS (Debug store)                 ACPI (ACPI supported)                 MMX (MMX technology supported)                 FXSR (FXSAVE and FXSTOR instructions supported)                 SSE (Streaming SIMD extensions)                 SSE2 (Streaming SIMD extensions 2)                 SS (Self-snoop)                 HTT (Multi-threading)                 TM (Thermal monitor supported)                 PBE (Pending break enabled)         Version: Intel(R) Xeon(R) CPU           X5670  @ 2.93GHz         Voltage: 1.2 V         External Clock: 6400 MHz         Max Speed: 4400 MHz         Current Speed: 2930 MHz         Status: Populated, Enabled         Upgrade: ZIF Socket         L1 Cache Handle: 0x0002         L2 Cache Handle: 0x0003         L3 Cache Handle: 0x0004         Serial Number: Not Specified         Asset Tag: Not Specified         Part Number: Not Specified         Core Count: 6         Core Enabled: 6         Thread Count: 12         Characteristics:                 64-bit capable Handle 0x0005, DMI type 4, 40 bytes Processor Information         Socket Designation: Node 1 Socket 2         Type: Central Processor         Family: Xeon MP         Manufacturer: Intel(R) Corporation         ID: C2 06 02 00 FF FB EB BF         Signature: Type 0, Family 6, Model 44, Stepping 2         Flags:                 FPU (Floating-point unit on-chip)                 VME (Virtual mode extension)                 DE (Debugging extension)                 PSE (Page size extension)                 TSC (Time stamp counter)                 MSR (Model specific registers)                 PAE (Physical address extension)                 MCE (Machine check exception)                 CX8 (CMPXCHG8 instruction supported)                 APIC (On-chip APIC hardware supported)                 SEP (Fast system call)                 MTRR (Memory type range registers)                 PGE (Page global enable)                 MCA (Machine check architecture)                 CMOV (Conditional move instruction supported)                 PAT (Page attribute table)                 PSE-36 (36-bit page size extension)                 CLFSH (CLFLUSH instruction supported)                 DS (Debug store)                 ACPI (ACPI supported)                 MMX (MMX technology supported)                 FXSR (FXSAVE and FXSTOR instructions supported)                 SSE (Streaming SIMD extensions)                 SSE2 (Streaming SIMD extensions 2)                 SS (Self-snoop)                 HTT (Multi-threading)                 TM (Thermal monitor supported)                 PBE (Pending break enabled)         Version: Intel(R) Xeon(R) CPU           X5670  @ 2.93GHz         Voltage: 1.2 V         External Clock: 6400 MHz         Max Speed: 4400 MHz         Current Speed: 2930 MHz         Status: Populated, Enabled         Upgrade: ZIF Socket         L1 Cache Handle: 0x0006         L2 Cache Handle: 0x0007         L3 Cache Handle: 0x0008         Serial Number: Not Specified         Asset Tag: Not Specified         Part Number: Not Specified         Core Count: 6         Core Enabled: 6         Thread Count: 12         Characteristics:                 64-bit capable

Memory imformation using dmidecode on Linux

There are times when we would need to increase the physical memory of our Oracle server, or the NameNode of a Hadoop cluster. To do that, we would need to know the maximum memory that can be added to the server, and the available slots/banks.

dmidecode gives all the information that is needed to do exactly that.

dmidecode give us information like the
Size: 4096 MB, etc.
Form Factor eg. DIMM, LRDIMM, Micro-DIMM, etc.
Locator eg DIMM01, DIMM02, etc.
Bank Locator eg. BANK01, BANK02, etc.
Speed eg. 800 MHz, 333 MHz, etc.
Manufacturer eg. Hynix, 014F00000000, etc.

Below we see that the Linux system can have a Maximum Capacity of 288 GB and has 18 Memory banks (highlighted in green).
 Currently the system is using 16 of the banks, and each bank has 4 GB of DIMM (highlighted in yellow).
ie. Current Total Memory = 4 GB x 16 = 64 GB

The banks highlighted with red are the once that aren't being used by the system. So we know that to achieve the Maximum capacity we need 16 GB DIMMs in each of the 18 banks.
ie. 16 GB x 18 = 288 GB

[root@isvx3 match]# dmidecode > dmidecode.out

Below is the Memory/DMI information from the dmidecode output redirected to dmidecode.out

Handle 0x0009, DMI type 16, 15 bytes Physical Memory Array     Location: System Board Or Motherboard     Use: System Memory     Error Correction Type: Single-bit ECC     Maximum Capacity: 288 GB     Error Information Handle: Not Provided     Number Of Devices: 18 Handle 0x000A, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM01     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x000B, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM01     Bank Locator: BANK01     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 72DC350E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x000C, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM02     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x000D, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM02     Bank Locator: BANK02     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 53DCC50E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x000E, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM03     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x000F, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM03     Bank Locator: BANK03     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 5DDC250E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0010, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM04     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0011, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM04     Bank Locator: BANK04     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 71DC950E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0012, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM05     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0013, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM05     Bank Locator: BANK05     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 60DC350E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0014, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM06     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0015, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM06     Bank Locator: BANK06     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 44DC950E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0016, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM07     Bank Connections: 0 0     Current Speed: Unknown     Type: None     Installed Size: Not Installed     Enabled Size: Not Installed     Error Status: OK Handle 0x0017, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: Unknown     Data Width: Unknown     Size: No Module Installed     Form Factor: Unknown     Set: None     Locator: DIMM07     Bank Locator: BANK07     Type: Unknown     Type Detail: Unknown     Speed: Unknown     Manufacturer: Not Specified     Serial Number: Not Specified     Asset Tag: Not Specified     Part Number: Not Specified Handle 0x0018, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM08     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0019, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM08     Bank Locator: BANK08     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 54DC150E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x001A, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM09     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x001B, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM09     Bank Locator: BANK09     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 41DCA50E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x001C, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM10     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x001D, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM10     Bank Locator: BANK10     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 5DDC350E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x001E, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM11     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x001F, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM11     Bank Locator: BANK11     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 70DC650E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0020, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM12     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0021, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM12     Bank Locator: BANK12     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 5EDC850E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0022, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM13     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0023, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM13     Bank Locator: BANK13     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 47DCB50E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0024, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM14     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0025, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM14     Bank Locator: BANK14     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 45DC150E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0026, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM15     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x0027, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM15     Bank Locator: BANK15     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 42DC250E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x0028, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM16     Bank Connections: 0 0     Current Speed: Unknown     Type: None     Installed Size: Not Installed     Enabled Size: Not Installed     Error Status: OK Handle 0x0029, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: Unknown     Data Width: Unknown     Size: No Module Installed     Form Factor: Unknown     Set: None     Locator: DIMM16     Bank Locator: BANK16     Type: Unknown     Type Detail: Unknown     Speed: Unknown     Manufacturer: Not Specified     Serial Number: Not Specified     Asset Tag: Not Specified     Part Number: Not Specified Handle 0x002A, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM17     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x002B, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM17     Bank Locator: BANK17     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 60DC150E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9 Handle 0x002C, DMI type 6, 12 bytes Memory Module Information     Socket Designation: DIMM18     Bank Connections: 0 0     Current Speed: 1 ns     Type: DIMM     Installed Size: 4096 MB (Single-bank Connection)     Enabled Size: 4096 MB (Single-bank Connection)     Error Status: OK Handle 0x002D, DMI type 17, 27 bytes Memory Device     Array Handle: 0x0009     Error Information Handle: Not Provided     Total Width: 72 bits     Data Width: 64 bits     Size: 4096 MB     Form Factor: DIMM     Set: None     Locator: DIMM18     Bank Locator: BANK18     Type: Other     Type Detail: Unknown     Speed: 800 MHz     Manufacturer: Hynix     Serial Number: 53DC950E     Asset Tag: Unknown     Part Number: HMT151R7BFR4C-H9

PERL odds and sods

Dumping some of the PERL scripts that I had written from my home directory into this blog entry

Array:
 array_count.pl

#!/usr/bin/perl -w # # This script simply gives the length of the array # @numbers = qw(one two three four); $array_length  = @numbers; print "Array length = $array_length\n";

array_reversal.pl

#!/usr/bin/perl -w # # This script give an example of how an array is reversed in perl # @teams = qw(chelsea liverpool mancity arsenal); foreach my $team (reverse @teams) {     print ("$team \n"); }

Capture command line output:
mysqld_port.pl

#!/usr/bin/perl -w # # This script redirects the mysql infor from netstat into a file # open (MYHANDLE, "netstat -tulpn|"); open (OUTHANDLE, ">> mysql.out"); while(<MYHANDLE>) {     if (/mysqld/) {         print OUTHANDLE $_;     } } close (OUTHANDLE); close (MYHANDLE);

num_files.pl

#!/usr/bin/perl -w # # This scripts counts the number of files in the current directory # sub file_count {     open (MYHANDLER,"ls -1 |");     $count = 0;     while (<MYHANDLER>) {         $file = $_;         chomp($file);         if (-f $file) {             $count = $count + 1;         }     }     print "There are $count files in the directory\n"; } file_count();

open_netstat.pl

#!/usr/bin/perl -w # # This script redirects the output of netstat into a file # open (MYHANDLE, "netstat -tupl|"); open (MYOUT, ">>mynetstat.txt"); while(<MYHANDLE>) {     print MYOUT $_; }

File IO:
check_logfiles.pl

#!/usr/bin/perl -w # # This script accepts log files as input and checks for the string 'connection # accepted from' # while ($data = <>) {         if ($data =~ /\bconnection accepted from\b/) {                 print "$data";         } }

get_nameserver.pl

#!/usr/bin/perl -w # # This function get the name server ip addresses from the resolv.conf file # sub get_nameserver {     open(MYHANDLER, "/etc/resolv.conf") || die ("ERROR: unable to open file\n");     while(<MYHANDLER>) {         if(/nameserver\b/)         {             @ip_addr = split(/\s+/,$_);             open(MYOUT,">>outfile");             print OUT "$ip_addr[1]\n";     }    }    close(MYHANDLER);    close(MYOUT); } get_nameserver();

number_of_lines.pl

#!/usr/bin/perl -w # # This function asks the user to enter a file name # sub enter_filename {     print "Enter a file name: ";     $file_name = <STDIN>;     chomp($file_name);     return($file_name); } # # This function counts the number of lines in the file # sub number_lines {     $i = 0;     $file_name = $_[0];     if (-e $file_name) {         open (MYHANDLER, $file_name) || die("ERROR: unable to open file";         while (<MYHANDLER>)         {             $i=$i+1;         }         close(MYHANDLER);         return($i);     } else {         print "File $file_name does not exist.\n";     } } $user_file_name = enter_filename(); $number_of_lines = number_lines($user_file_name); print "Number of lines in the file: $number_of_lines\n";

read_hosts.pl

#!/usr/bin/perl -w $file_name = "/etc/hosts"; # # This function returns host names and ip address from the /etc/hosts file # sub host_ipaddr {     open(MYHANDLE,$file_name);     while(<MYHANDLE>)     {         chomp;         if (/^[0-9]/)         {             @fields = split(/\s+/,$_);             print ("$fields[0] $fields[1]\n");         }     }    close(MYHANDLE); } host_ipaddr($file_name);

read_passwd.pl

#!/usr/bin/perl -w # # This function reads the password file and outputs it onto the screen/stdout # sub read_passwd {     open (FILEHANDLE, "/etc/passwd") || dir("ERROR: unable to open file\n");     while ($name = <FILEHANDLE>)     {         chomp($name);         print "$name \n";     }     close (FILEHANDLE); } read_passwd();

remove_blank.pl

#!/usr/bin/perl -w # # This script removed the blank lines from the file passed to it. The out # gets displayed on the screen/stdout. # if (! defined $ARGV[0]) {     usage(); } else {     $my_file = "$ARGV[0]"; } if (-e $my_file) {     print ("File exists\n"); } else {     print ("File does not exists.\n");     exit 1; } open (MYHANDLER, $my_file) || dir("ERROR: unable to open file\n"); while(<MYHANDLER>) {      if (/[^\s+]/) {         print $_;     } } close(MYHANDLER); sub usage {     print ("USAGE: .\/remove_blank.pl <file_name>\n");     exit 1; }

write_passwd.pl

#!/usr/bin/perl -w # # This function reads the password file and outputs it onto the file outfile.txt # sub write_passwd {     open (FILEHANDLE, "/etc/passwd") || dir("ERROR: unable to open file\n");     open (OUTFILE, ">outfile.txt") || dir("ERROR: unable to open file\n");     while ($name = <FILEHANDLE>)     {         print OUTFILE $name;     }     close (FILEHANDLE);     close (OUTFILE); } write_passwd();

Loops:
print_num.pl

#!/usr/bin/perl -w print "Enter a number: "; $my_num = <STDIN>; chomp($my_num); print_number($my_num); sub print_number {     $prtnum = $_[0];     print "$prtnum\n";     for ($i=$prtnum-1; $i >=0 ; $i--) {         print_number($i);     } }

Match:
grep_user.pl

#!/usr/bin/perl -w while (<>) {  if (/oracle/)  {   print $_;  } }

number_range.pl

#!/usr/bin/perl -w sub user_interface {     print "Enter a number between 1 to 9: ";     $my_number = <STDIN>;     chomp ($my_number);     return $my_number; } sub print_number {     my $my_number = $_[0];     if (($my_number >= 1) && ($my_number <= 9))     {         print "The number entered is: $my_number\n";     } else {         print "The number $my_number is out of range.\n";     } } $num_entered = user_interface(); print_number($num_entered);

single_digit.pl

#!/usr/bin/perl -w $_ = 32; if (/[0123456789]/) {  print "Match: $_ \n";  }

Split:
greatest_number.pl

#!/usr/bin/perl -w sub user_interface {     print "Enter some numbers: ";     $numbers = <STDIN>;     chomp($numbers);     return $numbers; } sub greatest_number {     @number_array = split(/\s+/,$_[0]);     $x = $number_array[0];     foreach $i (@number_array) {         if ($i > $x) {             $x = $i;         }     }     print "The greatest number is $x\n"; } $num_entered = user_interface(); greatest_number($num_entered);

split_ipaddr.pl

#!/usr/bin/perl -w $ip_addr = "129.40.71.195"; @addr = split(/\./,$ip_addr); foreach $num (@addr) {     print "$num\n"; }

split.pl

#!/usr/bin/perl -w # # This script finds if user oracle exists in the /etc/passwd file # open (MYHANDLER,"/etc/passwd"); while (<MYHANDLER>) {     @fields = split(/:/,$_);     $_ = $fields[0];     if (/oracle/)     {         print "User oracle exists.\n";         print "$_\n";     } }

String:
reverse.pl

#!/usr/bin/perl -w $mystring = "hello"; $mylength = length($mystring); @fields = split(//,$mystring); for ($i=$mylength-1; $i>=0; $i--) {     print $fields[$i]; } print "\n";

strcmp.pl

#!/usr/bin/perl -w sub user_interface {     print "Enter the first string: ";     $strs = <STDIN>;     chomp($strs);     return ($strs); } sub string_compare {     print "string_compare function $_[0]\n";     @strings = split(/\s+/,$_[0]);     print "The strings after splitting: $strings[0] $strings[1]\n";     $len_one = length($strings[0]);     $len_two = length($strings[1]);     if($len_one != $len_two)     {         print "The length of the strings is different\n";     } else {         print "The length of the strings is same\n";         @first_array = split(//,$strings[0]);         @second_array = split(//,$strings[1]);         for ($i=0; $i < $len_one; $i++) {             if ($first_array[$i] ne $second_array[$i]) {                 print "The words are different\n";                 exit 1;             }         }         print "The words are the same\n";     } } $my_strings = user_interface(); print "The strings to be compared are $my_strings\n"; string_compare($my_strings); exit;

strcpy.pl

#!/usr/bin/perl -w $orig_string = $ARGV[0]; print ("Original: $orig_string \n"); $final_string = $orig_string; print ("Result: $final_string \n"); [root@isvx3 string_manipulation]# cat strcpy.pl #!/usr/bin/perl -w $orig_string = $ARGV[0]; print ("Original: $orig_string \n"); $final_string = $orig_string; print ("Result: $final_string \n");

stringlen.pl

#!/usr/bin/perl -w $myword = "silly me"; $mylen = length($myword); print "Length = $mylen";

string_reversal.pl

#!/usr/bin/perl -w $myword = "silly me"; $mylen = length($myword); print "Length = $mylen";

upper_lower.pl

#!/usr/bin/perl -w print "Enter a world: "; $myword = <STDIN>; chomp($myword); $myword =~ s/\W.*//; print "The word entered is: $myword \n";

user_exists.pl

#!/usr/bin/perl -w $file = $ARGV[0]; if (-e $file) { open(MYHANDLER, $file); while(<MYHANDLER>) {     chomp;     if (/\bHadoop\b/)     {         print "$_ \n";     } } close(MYHANDLER); } else {     print "File $file does not exist\n"; }

wordcountmap.pl

#!/usr/bin/perl -w while (<STDIN>) {     chomp;     @words = split(/\s+/,$_);     foreach $w (@words) {         print "$w\n";     } }