home/Data quality/Pandora/Get started/Technical recommendations

Technical recommendations

Experian Pandora is a self-contained client/server product which runs on most java-compliant operating systems oncommodity hardware . It takes full advantage of 64-bit architectures, and is multi-threaded and linearly scalable.

Experian Pandora can be installed as a full client/server or multi-user installation, please view our installation guides for Windows and Linux once you have acquired the relevant hardware for your business needs.

Virus Checkers

We strongly recommend that you ensure that any anti-virus software installed does not check the directories containing Experian Pandora data files, and any system sweeps are scheduled to run outside of office hours, or periods where data loading will occur. This is to avoid any performance drops that might happen during critical operations.

System recommendations

These are recommendations and as such should be utilized as a reference point, all specifications provided don't have to be followed to the letter but rather used as a way to determine the type of component needed to obtain an acceptable level of performance for the range of business uses in question.

They will be provided in four types based on common usage levels: Minimum, Small, Medium and Large workloads.

Minimum requirements

Important: These are the minimum specifications that will allow you to make use of Experian Pandora; however, we don't recommend running Experian Pandora as we're unable to guarantee that you will see acceptable performance unless you're a single user running the product on a single machine.

Minimum server
Component Minimum
Operating System (OS) Windows 64-bit (7 and 10, Server 2012 R2 and Server 2016), CentOS Linux 64-bit, Red Hat Linux 64-bit, Ubuntu Linux 64-bit
Processor (CPU) Intel Core i3 2ghz Dual Core (Intel i3 6100) 1 User
Memory (RAM) 8GB available for Experian Pandora and OS (any running 1333Mhz)
Disk (HDD, SSD etc) 7,200rpm HDD for OS and any SSD for Experian Pandora
Network Card Gigabit Ethernet
Minimum client
Component Minimum
Operating System (OS) Windows 32-bit or 64-bit (7 and 10)
Processor (CPU) Intel Core i3 2ghz Dual Core (Intel i3 6100)
Memory (RAM) 1GB
Disk (HDD, SSD etc) Any HDD ~300Mb free space for client install
Network Card Gigabit Ethernet

Small workload

This would typically be for 1-3 or 1-4 users

Server recommendation
Component Recommended
Operating System (OS) Windows 64-bit (7 and 10, Server 2012 R2 and Server 2016), CentOS Linux 64-bit, Red Hat Linux 64-bit, Ubuntu Linux 64-bit
Processor (CPU) Intel Core i7 4.0ghz Quad Core (Intel i7 6700K) 1-3 Users Intel Xeon 3.5ghz Six Core (Xeon E5-1650) 1-4 Users
Memory (RAM) 16GB available for Experian Pandora and OS (Any running 1600Mhz)
Disk (HDD, SSD etc) 7,200rpm HDD for OS and SAS Drives or SSDs for Experian Pandora
Network Card Gigabit Ethernet
Server clarifications

Processor

Fewer and slower cores will still function with a corresponding decrease in throughput and longer load times. The ability to support multiple users will also be reduced as a result.

Memory

This is based on the needs of the operating system and the Experian Pandora application. For larger workloads with many users, more memory will be required. Additional memory will provide additional performance benefits when loading very large datasets.

Disk

Experian Pandora will benefit from using the fastest hard drives possible, ideally 15,000rpm SAS drives or Enterprise Solid State Disks (SSDs). It is recommended that RAID-5 or RAID-10 be used for data storage, which gives both redundancy and parallel throughput. Temporary storage can use RAID-0.

Disk throughput should exceed 150Mb/s sustained with average disk seek below 8ms for optimal performance.

Client Recommendation
Component Recommended
Operating System (OS) Windows 32-bit or 64-bit (7 and 10)
Processor (CPU) Intel Core i3 2ghz Dual Core (Intel i3 6100)
Memory (RAM) 4GB
Disk (HDD, SSD etc) Any HDD ~300MB free space for client install
Network Card Gigabit Ethernet
Client clarifications

The Experian Pandora client adopts a thin-client approach in that it performs presentation only. The main requirement is sufficient display resolution to allow the end-user maximum productivity.

By default, the application uses maximum of 512MB memory. The remainder is required for the operating system. Power users may require 1GB of application memory for optimum performance. The 4GB recommended memory is specified in the table above as most PCs come with at least that as standard, which provides plenty for the OS to utilize.

Medium workload

This would typically be for 3-6 users

Server recommendation
Component Recommended
Operating System (OS) Windows 64-bit (Server 2012 R2 and Server 2016), CentOS Linux 64-bit, Red Hat Linux 64-bit, Ubuntu Linux 64-bit
Processor (CPU) Intel Xeon 2.6ghz Eight Core (Intel Xeon E5-2640) 3-6 Users Dual Intel Xeon 3.5Ghz Six Core (2x Intel Xeon E5-1650) 3-6 Users
Memory (RAM) 32GB available for Experian Pandora and OS (any running 2133mhz)
Disk (HDD, SSD etc) Multiple SAS Drives or Enterprise SSDs setup with Experian Pandora on a separate drive to the OS
Network Card Gigabit Ethernet
Server clarifications

Processor

Fewer and slower cores will still function with a corresponding decrease in throughput and longer load times. The ability to support multiple users will also be reduced as a result.

Memory

This is based on the needs of the operating system and the Experian Pandora application. For larger workloads with many users, more memory will be required. Additional memory will provide additional performance benefits when loading very large datasets.

Disk

Experian Pandora will benefit from using the fastest hard drives possible, ideally 15,000rpm SAS drives or Enterprise Solid State Disks (SSDs). It is recommended that RAID-5 or RAID-10 be used for data storage, which gives both redundancy and parallel throughput. Temporary storage can use RAID-0.

Disk throughput should exceed 150Mb/s sustained with average disk seek below 8ms for optimal performance.

Client recommendation
Component Recommended
Operating System (OS) Windows 32-bit or 64-bit (7 and 10)
Processor (CPU) Intel Core i3 2ghz Dual Core (Intel i3 6100)
Memory (RAM) 4GB
Disk (HDD, SSD etc) Any HDD ~300Mb free space for client install
Network Card Gigabit Ethernet
Client clarifications

The Experian Pandora client adopts a thin-client approach in that it performs presentation only. The main requirement is sufficient display resolution to allow the end-user maximum productivity.

By default, the application uses maximum of 512MB memory. The remainder is required for the operating system. Power users may require 1GB of application memory for optimum performance. The 4GB recommended memory is specified in the table above as most PCs come with at least that as standard, which provides plenty for the OS to utilize.

Large workload

This would typically be for 5 or more users

Server recommendation
Component Recommended
Operating System (OS) Windows 64-bit (Server 2012 R2 and Server 2016), CentOS Linux 64-bit, Red Hat Linux 64-bit, Ubuntu Linux 64-bit
Processor (CPU) Dual Intel Xeon 2.6ghz Eight Core (2x Intel Xeon E5-2640) 5+ Users
Memory (RAM) 120GB available for Experian Pandora and OS (any running 2133mhz)
Disk (HDD, SSD etc) 4 x 400GB 12G SAS SSDs (RAID 0, for Experian Pandora data), total of around 1.5TB 2 x 300GB 6G SAS 15k rpm HDDs (mirrored, for OS), partitioned to 60GB (OS), 160GB (Temp)
Network Card Gigabit Ethernet
Server clarifications

Processor

Fewer and slower cores will still function with a corresponding decrease in throughput and longer load times. The ability to support multiple users will also be reduced as a result.

Memory

This is based on the needs of the operating system and the Experian Pandora application. For larger workloads with many users, more memory will be required. Additional memory will provide additional performance benefits when loading very large datasets.

Disk

Experian Pandora will benefit from using the fastest hard drives possible, ideally 15,000rpm SAS drives or Enterprise Solid State Disks (SSDs). It is recommended that RAID-5 or RAID-10 be used for data storage, which gives both redundancy and parallel throughput. Temporary storage can use RAID-0.

Disk throughput should exceed 150Mb/s sustained with average disk seek below 8ms for optimal performance.

Client recommendation
Component Recommended
Operating System (OS) Windows 32-bit or 64-bit (7 and 10)
Processor (CPU) Intel Core i3 2ghz Dual Core (Intel i3 6100)
Memory (RAM) 4GB
Disk (HDD, SSD etc) Any HDD ~300Mb free space for client install
Network Card Gigabit Ethernet
Client clarifications

The Experian Pandora client adopts a thin-client approach in that it performs presentation only. The main requirement is sufficient display resolution to allow the end-user maximum productivity.

By default, the application uses maximum of 512MB memory. The remainder is required for the operating system. Power users may require 1GB of application memory for optimum performance. The 4GB recommended memory is specified in the table above as most PCs come with at least that as standard, which provides plenty for the OS to utilize.

Storage devices and services

You will commonly hear or be concerned about the following terms with storage for Experian Pandora:

  • IOPS - how often or fast the storage device can perform I/O requests. IOPS from Experian Pandora standpoint isn't a particularly useful measure due to a lot of random access to the repository. It depends on too many parameters to be practically useful – the actual data, the rules, the number of users, the volumes and desired processing times, the operations carried out, etc.
  • latency - how long it takes for an I/O request to begin, measured in ms (milliseconds)
  • throughput - the actual speed of the data transfer, most often measured in MB/s (megabytes per second)

Comparison table

Type Average throughput in MB/s Average latency in ms (read/write)
HDD 150 – 600 <9 / <5
SSD 500 – 2400 0.5 / 0.1
SAS HDD 300 – 1500 <9 / <0.9

See below for further information on some of the storage types available.

HDD (Hard Disk Drive)

Standard hard disk drives are not something we recommend, this is purely from a performance standpoint. The only time you'd be happy with one is if you do very little work that is not of a time sensitive or critical nature. These typically come with 5400rpm or 7200rpm, which dictates how quick they can retrieve data on their disks.

SSD (Solid State Drive)

Standard solid state drives are again recommended for light workloads due to the finite lifespan. We do recommend Enterprise level SSDs as they provide the performance of a standard SSD but have the purpose built life span required for a server based environment and can deal with both high volume and time sensitive data.

Expect to see average performances of 500MB/s – 2400MB/s throughput with read/write latencies (response times) consistently below 0.5ms and 0.1ms respectively.

SAS HDD (Serial Attached SCSI Hard Disk Drive)

These are basically Enterprise level HDDs that are developed with 24/7 uptime in mind and typically come in 10k or 15k rpm speeds. We recommend these for systems that Enterprise SSDs are not convenient for. The 15k SAS HDDs will provide you a vastly improved performance over normal consumer HDDs. This includes being built with servers and high workloads in mind.

Expect to see average performances of 300MB/s – 1500MB/s throughput with read/write latencies roughly in the figure of < 9ms and < 0.9ms respectively.

SAN (Storage Area Network)

This allows each server to access shared storage as if it were a drive directly attached to the server. Unfortunately, due the many variables in creating a SAN we can't give conclusive performance results. Something to be aware of for reference though is as follows:

A SAN is not a monolithic entity. If you take a switch-based Fibre-Channel SAN as an example, the I/O path that is considered part of the SAN starts from the host bus adapters (HBA) all the way to the eventual disk media on the disk array. Hardware components on this I/O path typically include switches, inter-switch links, front-side adapters, disk array cache and processors, disk controllers, and disk drive media. In addition, there are layers of software on this path, including various drivers, firmware, and APIs. Every single component on this I/O path has the potential to significantly alter the performance characteristics of a drive presented from the SAN.

The above is a statement from the blog at_: http://www.sqlteam.com/article/which-is-faster-san-or-directly-attached-storage#sthash.RRvlQT9W.dpuf_

We share similar views and based on that we are unable to recommend SANs because their implementation involves too many variables, many of which might be completely out of the hands of us or end-users.

Virtual machine considerations

Experian Pandora can be happily run on virtual machines with smaller datasets. Anything for larger datasets or intensive work it's worth noting that certain things need to be taken into account if you hope to get maximum performance from its full capabilities.

  • The VM must be configured with a fixed MAC address and not a dynamically set one, otherwise the license will stop working due to its MAC requirements.
  • It should be allocated with CPUs that are dedicated to it. This makes sure full performance is available to Experian Pandora rather than it getting potentially lower performance that comes with having the CPUs resources being shared across multiple VMs or applications on the server.
  • Directly attached storage for the VM if possible. Depending on your VM configuration this will be unlikely. Most VMs are used as quick builds based on sharing hardware resources. This can have negative impacts on performance for a database storage engine like Experian Pandora. It can be slower but more importantly it allows for other applications on the server to hinder performance.
  • If using a VM/SAN setup, it's advisable to double the recommended memory, this enables more memory allocation for caching that helps to offset the SAN. As stated above however we do not recommend SANs.