Adrian Silasi – SiSoftware

Intel Core i9 9900K CofeeLake-R Review & Benchmarks – CPU 8-core/16-thread Performance

By Adrian Silasi | 21st November 2018 - 12:00 am |22nd November 2018 Reviews

Comments Off

What is “CofeeLake-R” CFL-R?

It is the “refresh” (updated) version of the 8th generation Intel Core architecture (CFL) – itself a minor stepping of the previous 7th generation “KabyLake” (KBL), itself a minor update of the 6th generation “SkyLake” (SKL). While ordinarily this would not be much of an event – this time we do have more significant changes:

Patched vulnerabilities in hardware: this can help restore I/O workload performance degradation due to OS mitigations
- Kernel Page Table Isolation (KPTI) aka “Meltdown” – Patched in hardware
- L1TF/Foreshadow – Patched in hardware
- (IBPB/IBRS) “Spectre 2” – OS mitigation needed
- Speculative Store Bypass disabling (SSBD) “Spectre 4” – OS mitigation needed
Increased core counts yet again: CFL-R top-end now has 8 cores, not 6.

Intel CPUs bore the brunt of the vulnerabilities disclosed at the start of 2018 with “Meltdown” operating system mitigations (KVA) likely having the biggest performance impact in I/O workloads. While modern features (e.g. PCID (process context id) acceleration) could help reduce performance impact somewhat on recent architectures (4th gen and newer) the impact can still be significant. The CFL-R hardware fixes (thus not needing KVA) may thus prove very important.

On the desktop we also see increased cores (again!) now up to 8 (thus 16 threads with HyperThreading) – double what KBL and SKL brought and matching AMD.

We also see increased clocks, mainly Turbo, but this still allows 1 or 2 cores to boost clocks higher than CFL could and thus help workloads not massively threaded. This can improve responsiveness as single tasks can be run at top speed when there is little thread utilization.

While rated TDP has not changed, in practice we are likely to see increased “real” power consumption especially due to higher clocks – with Turbo pushing power consumption even higher – close to SKL/KBL-X.

In this article we test CPU Core performance; please see our other articles on:

Intel Core i7 9900K CofeeLake-R Review & Benchmarks – Memory Performance
Intel Core i7 8700K CofeeLake Review & Benchmarks – Memory Performance

Hardware Specifications

We are comparing the top-of-the-range Gen 8 Core i9 (9900K) with previous generation (8700K) and competing architectures with a view to upgrading to a mid-range high performance design.

CPU Specifications	Intel i9-9900K CofeeLake-R	Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Comments
Cores (CU) / Threads (SP)	8C / 16T	6C / 12T	8C / 16T	10C / 20T	We have 33% more cores matching Ryzen and close to mainstream SKL-X!
Speed (Min / Max / Turbo)	0.8-3.6-5GHz (8x-36x-50x)	0.8-3.7-4.7GHz (8x-37x-47x)	2.2-3.7-4.2GHz (22x-37x-42x)	1.2-3.3-4.3 (12x-33x-43x)	Single/Dual core Turbo has now reached 5GHz same as 8086K special edition.
Power (TDP)	95W (135)	95W (131)	105W (135)	140W (308)	TDP is the same but overall power consumption likely far higher.
L1D / L1I Caches	8x 32kB 8-way / 8x 32kB 8-way	6x 32kB 8-way / 6x 32kB 8-way	8x 32kB 8-way / 8x 64kB 8-way	10x 32kB 8-way / 10x 32kB 8-way	No change in L1 caches. Just more of them.
L2 Caches	8x 256kB 8-way	6x 256kB 8-way	8x 512kB 8-way	10x 1MB 8-way	No change in L2 caches. Just more of them.
L3 Caches	16MB 16-way	12MB 16-way	2x 8MB 16-way	13.75MB 11-way	L3 has also increased by 33% in line with cores matching Ryzen.
Microcode/Firmware	MU069E0C-9E	MU069E0A-96	MU8F0802-04	MU065504-49	We have a new stepping and slightly newer microcode.

Native Performance

We are testing native arithmetic, SIMD and cryptography performance using the highest performing instruction sets (AVX2, AVX, etc.). CFL supports most modern instruction sets (AVX2, FMA3) but not the latest SKL/KBL-X AVX512 nor a few others like SHA HWA (Atom, Ryzen).

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64 (1809), latest drivers. 2MB “large pages” were enabled and in use. Turbo / Boost was enabled on all configurations.

Spectre / Meltdown Windows Mitigations: all were enabled as per default (BTI enabled, RDCL/KVA enabled, PCID enabled).

Native Benchmarks		Intel i9-9900K CofeeLake-R	Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Comments

	Native Dhrystone Integer (GIPS)	400 [+20%]	291	334	485	In the old Dhrystone integer workload, CFL-R finally beats Ryzen by 20%.
	Native Dhrystone Long (GIPS)	393 [+17%]	296	335	485	With a 64-bit integer workload – nothing much changes.
	Native FP32 (Float) Whetstone (GFLOPS)	236 [+19%]	170	198	262	Switching to floating-point, CFL-R still beats Ryzen by 20% in old Whetstone.
	Native FP64 (Double) Whetstone (GFLOPS)	196 [+16%]	143	169	223	With FP64 nothing much changes.
From integer workloads in Dhrystone to floating-point workloads in Whetstone, CFL-R handily beats Ryzen by about 20% and is also 33-39% faster than old CFL. It’s “king of the hill”.

	Native Integer (Int32) Multi-Media (Mpix/s)	1000 [+74%]	741	574	1590 (AVX512)	In this vectorised AVX2 integer test CFL-R is almost 2x faster than Ryzen
	Native Long (Int64) Multi-Media (Mpix/s)	416 [+122%]	305	187	581 (AVX512)	With a 64-bit AVX2 integer vectorised workload, CFL-R is now over 2.2x faster than Ryzen.
	Native Quad-Int (Int128) Multi-Media (Mpix/s)	6.75 [+16%]	4.9	5.8	7.6	This is a tough test using Long integers to emulate Int128 without SIMD: still CFL-R is fastest.
	Native Float/FP32 Multi-Media (Mpix/s)	927 [+56%]	678	596	1760 (AVX512)	In this floating-point AVX/FMA vectorised test, CFL-R is 56% faster than Ryzen.
	Native Double/FP64 Multi-Media (Mpix/s)	544 [+62%]	402	335	533 (AVX512)	Switching to FP64 SIMD code, CFL-R increases its lead to 612.
	Native Quad-Float/FP128 Multi-Media (Mpix/s)	23.3 [+49%]	16.7	15.6	40.3 (AVX512)	In this heavy algorithm using FP64 to mantissa extend FP128 but not vectorised – CFL-R is 50% faster.
In vectorised SIMD code we know Intel’s SIMD units (that can execute 256-bit instructions in one go) are much more powerful than AMD’s and it shows: Ryzen is soundly beaten by a big 50-100% margin. Naturally it cannot beat its “big brother” SKL-X with AVX512 – which is likely why Intel has not enabled them.

	Crypto AES-256 (GB/s)	17.6 [+9%]	17.8	16.1	23	With AES HWA support all CPUs are memory bandwidth bound; core contention (8 vs 6) means CFL-R scores slightly worse than CFL.
	Crypto AES-128 (GB/s)	17.6 [+9%]	17.8	16.1	23	What we saw with AES-256 just repeats with AES-128.
	Crypto SHA2-256 (GB/s)	12.2 [-34%]	9	18.6	26 (AVX512)	With SHA HWA Ryzen2 powers through but CFL-R is only 34% slower.
	Crypto SHA1 (GB/s)	23 [+19%]	17.3	19.3	38 (AVX512)	Ryzen also accelerates the soon-to-be-defunct SHA1 but the algorithm is less compute heavy allowing CFL-R to beat it.
	Crypto SHA2-512 (GB/s)	9 [+139%]	6.65	3.77	21 (AVX512)	SHA2-512 is not accelerated by SHA HWA, allowing CFL-R to use its SIMD units and be 139% faster.
AES HWA is memory bound and here and CFL-R also enjoys the 3200Mt/s memory – but now feeding 8C / 16T which all contend for the bandwidth. Thus CFL-R does score slighly less than CFL and obviously gets left in the dust by SKL-X with 4 memory channels. Ryzen2 SHA HWA does manage a lonely win but anything SIMD accelerated belongs to Intel.

	Black-Scholes float/FP32 (MOPT/s)	276 [+7%]	207	257	309	In this non-vectorised test CFL-R is just a bit faster than Ryzen2.
	Black-Scholes double/FP64 (MOPT/s)	240 [+10%]	180	219	277	Switching to FP64 code, nothing much changes, CFL-R is 10% faster
	Binomial float/FP32 (kOPT/s)	59.9 [-44%]	47	107	70.5	Binomial uses thread shared data thus stresses the cache & memory system; CFL-R strangely loses by 44%.
	Binomial double/FP64 (kOPT/s)	61.9 [+2%]	44.2	60.6	68	With FP64 code Ryzen2’s lead diminishes, CFL is pretty much tied with it.
	Monte-Carlo float/FP32 (kOPT/s)	56.5 [+4%]	41.6	54.2	63	Monte-Carlo also uses thread shared data but read-only thus reducing modify pressure on the caches; CFL-R is just 4% faster.
	Monte-Carlo double/FP64 (kOPT/s)	44.3 [+8%]	32.9	41	50.5	Switching to FP64 CFL-R increases its lead to 8%.
Without SIMD support, CFL-R relies on its thread count increase (thus matching Ryzen2) to beat Ryzen2 by a small amount and lose in one test. But in a test that AMD used to always win (with Ryzen 1/2) Intel now has the lead.

	SGEMM (GFLOPS) float/FP32	403 [+34%]	385	300	413 (AVX512)	In this tough vectorised AVX2/FMA algorithm CFL-R is 35% faster than Ryzen2
	DGEMM (GFLOPS) double/FP64	269 [+126%]	135	119	212 (AVX512)	With FP64 vectorised code, CFL-R is over 2x faster.
	SFFT (GFLOPS) float/FP32	23.4 [+160%]	24	9	28.6 (AVX512)	FFT is also heavily vectorised (x4 AVX/FMA) but stresses the memory sub-system more CFL-R is over 2x faster.
	DFFT (GFLOPS) double/FP64	11.2 [+41%]	11.9	7.92	14.6 (AVX512)	With FP64 code, CFL-R’s lead reduces to 41%.
	SNBODY (GFLOPS) float/FP32	550 [+96%]	411	280	638 (AVX512)	N-Body simulation is vectorised but many memory accesses to shared data but CFL-R is 2x faster than Ryzen2.
	DNBODY (GFLOPS) double/FP64	172 [+52%]	127	113	195 (AVX512)	With FP64 code CFL’s lead reduces to 50% over Ryzen 2.
With highly vectorised SIMD code CFL-R performs very well – dominating Ryzen2: in some tests it is over 2x faster! Then again CFL did not have any issues here either, Intel is just extending their lead…

	Blur (3×3) Filter (MPix/s)	2270 [+86%]	1700	1220	4540 (AVX512)	In this vectorised integer AVX2 workload CFL-R is amost 2x faster than Ryzen2.
	Sharpen (5×5) Filter (MPix/s)	903 [+67%]	675	542	1790 (AVX512)	Same algorithm but more shared data reduces the lead to 67% still significant.
	Motion-Blur (7×7) Filter (MPix/s)	488 [+61%]	362	303	940 (AVX512)	Again same algorithm but even more data shared reduces the lead to 61%.
	Edge Detection (2*5×5) Sobel Filter (MPix/s)	784 [+73%]	589	453	1520 (AVX512)	Different algorithm but still AVX2 vectorised workload means CFL-R is 73% faster than Ryzen 2.
	Noise Removal (5×5) Median Filter (MPix/s)	78.6 [+13%]	57.8	69.7	223 (AVX512)	Still AVX2 vectorised code but CFL-R stumbles a bit here – but it’s still 13% faster.
	Oil Painting Quantise Filter (MPix/s)	43.4 [+76%]	31.8	24.6	70.8 (AVX512)	CFL-R recovers its dominance over Ryzen2.
	Diffusion Randomise (XorShift) Filter (MPix/s)	4470 [+208%]	3480	1450	3570 (AVX512)	CFL-R (like all Intel CPUs) does very well here – it’s a huge 3x faster.
	Marbling Perlin Noise 2D Filter (MPix/s)	614 [+153%]	448	243	909 (AVX512)	In this final test, CFL-R is over 2x faster than Ryzen 2.

Adding 2 more cores brings additional performance gains (not to mention the higher Turbo clock) over CFL which showed big gains over the old SKL/KBL again within the same (rated) TDP. Intel never had any problem with SIMD code (AVX/AVX2/FMA3) beating Ryzen2 by a large margin (now over 2x faster) but now also wins pretty much all tests.

It is consistently 33-40% faster than CFL (8700K) in line with core/speed increases which bodes well for compute-heavy code; streaming performance can be lower due to increase core contention for bandwidth and here faster (though more expensive) memory would help.

No – it cannot beat its “older big brother” SKL-X with AVX512 – not to mention increased core/thread count as well as memory channels, but in some tests it is competitive.

Software VM (.Net/Java) Performance

We are testing arithmetic and vectorised performance of software virtual machines (SVM), i.e. Java and .Net. With operating systems – like Windows 10 – favouring SVM applications over “legacy” native, the performance of .Net CLR (and Java JVM) has become far more important.

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64 (1809), latest drivers. 2MB “large pages” were enabled and in use. Turbo / Boost was enabled on all configurations.

Spectre / Meltdown Windows Mitigations: all were enabled as per default (BTI enabled, RDCL/KVA enabled, PCID enabled).

VM Benchmarks		Intel i9-9900K CofeeLake-R	Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Comments

	.Net Dhrystone Integer (GIPS)	54.3 [-11%]	41	61	52	.Net CLR integer performance starts off well but CFL-R is 10% slower.
	.Net Dhrystone Long (GIPS)	55.8 [-7%]	41	60	54	With 64-bit integers the gap lowers to 7%.
	.Net Whetstone float/FP32 (GFLOPS)	95.5 [-6%]	78	102	107	Floating-Point CLR performance does not change much, CFL-R is still 6% slower than Ryzen2 despite big gain over old CFL/KBL/SKL.
	.Net Whetstone double/FP64 (GFLOPS)	126 [+8%]	95	117	137	FP64 performance allows CFL-R to win in the end.
Ryzen 2 performs exceedingly well in .Net workloads – but CFL-R can hold its own and overall it is not significantly slower (under 10%) and even wins 1 test out of 4.

	.Net Integer Vectorised/Multi-Media (MPix/s)	144 [+30%]	93.5	111	144	Unlike CFL, here CFL-R is 30% faster than Ryzen2.
	.Net Long Vectorised/Multi-Media (MPix/s)	139 [+28%]	93.1	109	143	With 64-bit integer workload nothing much changes.
	.Net Float/FP32 Vectorised/Multi-Media (MPix/s)	499 [+27%]	361	392	585	Here we make use of RyuJit’s support for SIMD vectors thus running AVX/FMA code and CFL-R is 27% faster than Ryzen2.
	.Net Double/FP64 Vectorised/Multi-Media (MPix/s)	274 [+26%]	198	217	314	Switching to FP64 SIMD vector code – still running AVX/FMA – CFL-R is still faster.
We see a big improvement in CFL-R even against old CFL: this allows it to soundly beat Ryzen 2 (which used to win this test) by about 30%, a significant margin. It is possible the hardware fixes (“Meltdown”) are having an effect here.

	Java Dhrystone Integer (GIPS)	614 [+7%]	557	573	877	Java JVM performance starts well with a 7% lead over Ryzen2.
	Java Dhrystone Long (GIPS)	644 [+16%]	488	553	772	With 64-bit integers, CFL-R doubles its lead to 16%.
	Java Whetstone float/FP32 (GFLOPS)	143 [+9%]	101	131	156	Floating-point JVM performance is similar – 9% faster.
	Java Whetstone double/FP64 (GFLOPS)	147 [+6%]	103	139	160	With 64-bit precision the lead drops to 6%.
While CFL-R improves by a good amount over CFL (which itself improved greatly over KBL/SKL) and now beats Ryzen2 by a good margin 7-16%.

	Java Integer Vectorised/Multi-Media (MPix/s)	147 [+30%]	100	113	140	Without SIMD acceleration we still see CFL-R 30% than Ryzen2 in this integer workload.
	Java Long Vectorised/Multi-Media (MPix/s)	142 [+41%]	89	101	152	Changing to 64-bit integer increases the lead to 41%.
	Java Float/FP32 Vectorised/Multi-Media (MPix/s)	91 [-6%]	64	97	98	With floating-point non-SIMD accelerated Ryzen2 is faster.
	Java Double/FP64 Vectorised/Multi-Media (MPix/s)	93 [+3%]	64	90	99	With 64-bit floatint-point precision CFL-R is back on top by just 3%.
With compute heavy vectorised code but not SIMD accelerated, CFL-R is still faster or at least ties with Ryzen2.

CFL-R now beats or at least matches Ryzen2 in VM tests the latter used to easily win. It may not have the lead native SIMD vectorised code allows it – but has no trouble keeping up – unlike its older SKL/KBL “brothers”.

SiSoftware Official Ranker Scores

Final Thoughts / Conclusions

While Intel had finally increased core counts with CFL (after many years) in light of new competition from AMD with Ryzen/2 – it still relied on fewer but more powerful (at least in SIMD) cores to compete. CFL-R finally brings core (8 vs 8) and thread (16 vs 16) parity with the competition to ensure domination.

And with few exceptions – that’s what it has achieved – 9900K is the fastest desktop CPU at this time (November 2018) and if you can afford it you can upgrade on the same, old, 200-series mainboards (though naturally 300-series is recommended). The performance improvement (33-40% over 8700K) is pretty significant to upgrade – again if you can afford it – considering it is a “mere refresh”.

The “Meltdown” fixes in hardware are also likely to bring big improvement in I/O workloads – or to be precise – restore the performance loss of OS mitigations (KVA) that have been deployed this year (2018). Still, in very rough terms, now you don’t have to decide between “speed” and “security” – though perhaps KVA should be used by default just in case any CPU (not just Intel) leaks information between user/kernel spaces by a yet undiscovered side-channel vulnerability.

But despite the “i9” moniker – don’t think you’re getting a workstation-class CPU on the cheap: SKL-X not only (still) has more cores/threads and 2x memory channels but also supports AVX512 beating it soundly. It will also be refreshed soon – sporting the same “Meltdown” in-hardware fixes. But again considering the costs (almost 2x) CFL-R is likely the performance/price winner on most workloads.

For now, on the desktop, the 9900K is “king of the hill”!

SiSoftware Sandra Titanium (2018) SP2c Update

By Adrian Silasi | 14th November 2018 - 9:29 pm |29th November 2018 Releases, Updates

Comments Off

We are pleased to announce SP2c (version 28.34 for Sandra Titanium (2018) with updated hardware and software support:

Sandra Titanium (2018) Press Release

Hardware Information
- Ryzen 1/2 & Threadripper: fix crash when Hyper-V / “Windows Core Isolation – Memory Integrity” (VBS) – are enabled
- Ryzen 2 & Threadripper: enabled display of Core temperature, voltage, current, power usage

Reviews using Sandra 2018 SP2c

Intel Core i9 9900K CofeeLake-R Review & Benchmarks – CPU 8-core/16-thread Performance

Commercial version customers can download the free updates from their software distributor; Lite users please download from your favourite download site.

Download Sandra Lite

Black Friday 2018 Deal

Black Friday 2018 Deal

Intel Core i7 8700K CofeeLake Review & Benchmarks – UHD 630 GPGPU Performance

By Adrian Silasi | 30th October 2018 - 12:47 am |30th October 2018 Reviews

Comments Off

What is “CofeeLake” CFL?

The 8th generation Intel Core architecture is code-named “CofeeLake” (CFL): unlike previous architectures, it is a minor stepping of the previous 7th generation “KabyLake” (KBL), itself a minor update of the 6th generation “SkyLake” (SKL). As before, the CPUs contain an integrated GPU (with compute support aka GPGPU).

While originally Intel integrated graphics were not much use – starting with SNB (“SandyBridge”) and especially its GPGPU-capable successor IVB (“IvyBridge”) the integrated graphics units made large progress, with HSW (“Haswell”) introducing powerful many compute units (GT3+) and esoteric L4 cache (eDRAM) versions (“CrystallWell) supporting high-end features like FP64 (native 64-bit floating-point support) and zero-copy CPU <> GPU transfers.

Alas, while the features remained, the higher-end versions (GT3, GT4e) never became mainstream and pretty much disappeared – except very high-end ULV/H SKUs with top-end desktop CPUs like 6700K, 8700K, etc. tested here stuck with the low-end GT2 versions. Perhaps nobody in their right mind would use such CPUs without a dedicated external (GP)GPU, it is still interesting to see how the GPU core has evolved in time.

Also let’s not forget that on the mobile platforms (either ULV/Y even H) most laptops/tablets do not have dedicated GPU and rely solely on integrated graphics – and here naturally UHD630 performance matters.

Hardware Specifications

We are comparing the graphics units of to-of-the-range Intel CPUs with low-end dedicated cards to determine whether they are good enough for modest use, especially for compute (GPGPU) use supporting the CPU.

GPGPU Specifications	Intel UHD 630 (8700K)	Intel HD 530 (6700K)	nVidia GT 1030	Comments
Arch Chipset	GT2 / EV9.5	GT2 / EV9	GP108 / SM6.1	UHD6xx is just a minor revision of the HD5xx video core.
Cores (CU) / Threads (SP)	24 / 192	24 / 192	3 / 384	No change in core / SP units.
ROPs / TMUs	8 / 16	8 / 16	16 / 24	No change in ROP/TMUs either.
Speed (Min-Turbo)	350-1200	350-1150	300-1.26-1.52	Turbo speed is only slightly increased.
Power (TDP)	95W	91W	35W	TDP has gone up a bit but nothing major.
Constant Memory	3.2GB	3.2GB	64kB (dedicated)	There is no dedicated constant memory thus a large chunk is available to use (GB) unlike a dedicated video card with very fast but small (kB).
Shared (Local) Memory	64kB	64kB	48kB (dedicated)	Bigger than usual shared/local memory but slow (likely non dedicated).
Global Memory	7GB (of 16GB)	7GB (of 16GB)	2GB	About 50% of main memory can be used as global memory – thus pretty large workloads can be run.
Memory System	DDR4 3200Mt/s 128-bit	DDR4 2533Mt/s 128-bit	GDDR5 6Gt/s 64-bit	CFL can reliably run at faster data rates thus 630 benefits too.
Memory Bandwidth (GB/s)	50	40	48	The high data rate of DDR4 can result in higher bandwidth than some dedicated cards.
L2 Cache	512kB	512kB	48kB	L2 is unchanged and reasonably large.
FP64/double ratio	Yes, 1/8	Yes, 1/8	Yes, 1/32	FP64 is supported and at good ration compared to gimped dedicated cards.
FP16/half ratio	Yes, 2x	Yes, 2x	Yes, 1/64	FP16 is also now supported at twice the rate – again unlike gimped dedicated cards.

Processing Performance

We are testing both OpenCL performance using the latest SDK / libraries / drivers from both Intel and competition.

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64, latest Intel drivers, OpenCL 2.x. Turbo / Boost was enabled on all configurations.

Processing Benchmarks		Intel UHD 630 (8700K)	Intel HD 530 (6700K)	nVidia GT 1030	Comments

	Mandel FP16/Half (Mpix/s)	1150 [+7%]	1070	1660	Thanks to FP16 support we see double the performance over FP32 and thus only 50% slower than dedicated 1030.
	Mandel FP32/Single (Mpix/s)	584 [+9%]	535	1660	630 is almost 10% faster than old 530 but still about 1/3 of a dedicated 1030.
	Mandel FP64/Double (Mpix/s)	151 [+9%]	138	72.8	FP64 sees a similar delta (+9%) but much faster (2x) than a dedicated 1030 due to gimped FP64 units.
	Mandel FP128/Quad (Mpix/s)	7.84 [+5%]	7.46	2.88	Emulated FP128 precision depends entirely on FP64 performance and much better (3x) than gimped dedicated.
UHD630 is about 5-9% faster than 520, not much to celebrate – but due to native FP16 and especially FP64 support it can match or even overtake low-end dedicated GPUs – a pretty surprising result! If only we had more cores, it may actually be very much competitive.

	Crypto AES-256 (GB/s)	1 [+5%]	0.954	4.37	We see a 5% improvement for 630 0 but far lower performance than a dedicated GPU.
	Crypto AES-128 (GB/s)	1.3 [+6]	1.23	5.9	Nothing changes here , we see a 6% improvement.

	Crypto SHA2-256 (GB/s)	3.6 [+3%]	3.5	18.4	In this heavy integer workload, the improvement falls to just 3% – but a dedicated unit would be about 4x faster.
	Crypto SHA1 (GB/s)	8.18 [+2%]	8	24	Nothing much changes here, we see a 2% improvement.
	Crypto SHA2-512 (GB/s)	1.3 [+2%]	1.27	7.8	With 64-bit integer workload, same improvement of just 2% but now the 1030 is about 6x faster!
Nobody will be using integrated graphics for crypto-mining any time soon, we see a very minor improvement in 639 vs old 530, but overall low performance versus dedicated graphics like a 1030 which would be 4-6x faster. We would need 3x more cores to compete here.

	Black-Scholes float/FP32 (MOPT/s)	1180 [+21%]	977	1320	In this FP32 financial workload we see a good 21% improvement vs. old 530. Also good result vs. dedicated 1030.
	Black-Scholes double/FP64 (MOPT/s)	180 [+2%]	175	137	Switching to FP64 code, the difference is next to nothing but better than a gimped 1030.
	Binomial float/FP32 (kOPT/s)	111 [+12%]	99	255	Binomial uses thread shared data thus stresses the internal memory sub-system, and here 630 is 12% faster. But 1/2 the performance of a 1030.
	Binomial double/FP64 (kOPT/s)	22.3 [+4%]	21.5	14	With FP64 code the improvement drops to 4%.
	Monte-Carlo float/FP32 (kOPT/s)	298 [+2%]	291	617	Monte-Carlo also uses thread shared data but read-only thus reducing modify pressure – strangely we see only 2% improvement and again 1/2 1030 performance.
	Monte-Carlo double/FP64 (kOPT/s)	43.4 [+2%]	42.5	28	Switching to FP64 we see no changes. But almost 2x performance over a 1030.
You can run financial analysis algorithms with decent performance on an UHD630 – just as you could on the old 530 – and again better FP64 performance than dedicated – (GT 1030) a pretty impressive result. Naturally, you can just use the powerful CPU cores instead…

	SGEMM (GFLOPS) float/FP32	143 [+4%]	138	685	Using 32-bit precision 630 improves 4% but is almost 1/5 (5 times slower) than a 1030.
	DGEMM (GFLOPS) double/FP64	55.5 [+3%]	53.7	35	With FP64 precision, the delta does not change but now 640 is amost 2x faster than a 1030.
	SFFT (GFLOPS) float/FP32	39.6 [+20%]	33	37	FFT is memory access bound and here 630’s faster DDR4 memory gives it a 20% lead.
	DFFT (GFLOPS) double/FP64	9.3 [+16%]	8	20	We see a similar improvement with FP64 about 16%.
	SNBODY (GFLOPS) float/FP32	272 [+2%]	266	637	Back to normality with this algorithm – we see just 2% improvement.
	DNBODY (GFLOPS) double/FP64	27.7 [+3%]	26.9	32	With FP64 precision, nothing much changes.
The scientific scores are similar to financial ones – except the memory access heavy FFT which greatly benefits from better memory (if that is provided of course) but this a dedicated card (like the 1030) is much faster in FP32 mode but again the 630 can be 2x faster in FP64 mode. Again, you’re much better off using the CPU and its powerful SIMD units for these algorithms.

	Blur (3×3) Filter single/FP32 (MPix/s)	592 [+10%]	536	1620	In this 3×3 convolution algorithm, we see a 10% improvement over the old 530. But about 1/3x performance of a 1030.
	Sharpen (5×5) Filter single/FP32 (MPix/s)	128 [+9%]	117	637	Same algorithm but more shared data reduces the gap to 9%.
	Motion Blur (7×7) Filter single/FP32 (MPix/s)	133 [+9%]	122	391	With even more data the gap remains the same.
	Edge Detection (2*5×5) Sobel Filter single/FP32 (MPix/s)	127 [+9%]	116	368	Still convolution but with 2 filters – still 9% better.
	Noise Removal (5×5) Median Filter single/FP32 (MPix/s)	9.2 [+10%]	8.4	7.3	Different algorithm does not change much still 10% better.
	Oil Painting Quantise Filter single/FP32 (MPix/s)	10.6 [+9%]	9.7	4.08	Without major processing, 630 improves by the same amount.
	Diffusion Randomise (XorShift) Filter single/FP32 (MPix/s)	1640 [+2%]	1600	2350	This algorithm is 64-bit integer heavy thus we fall to the “usual” 2% improvement.
	Marbling Perlin Noise 2D Filter single/FP32 (MPix/s)	550 [+2%]	538	849	One of the most complex and largest filters, sees the same 2% improvement.
For image processing using FP32 precision 630 performs a bit better than usual, 10% faster across the board compared to the old 530 – but still about 1/3 (third) the speed of a dedicated 1030. But if you can make do with FP16 precision image processing, then we almost double performance.

Memory Performance

We are testing both OpenCL performance using the latest SDK / libraries / drivers from both Intel and competition.

Results Interpretation: Higher values (MB/s, etc.) mean better performance. Lower time values (ns, etc.) mean better performance.

Environment: Windows 10 x64, latest Intel drivers, OpenCL 2.x. Turbo / Boost was enabled on all configurations.

Memory Benchmarks		Intel UHD 630 (8700K)	Intel HD 530 (6700K)	nVidia GT 1030	Comments

	Internal Memory Bandwidth (GB/s)	36.4 [+21%]	30	38.5	Due to higher speed DDR4 memory, the 630 manages 21% better bandwidth than the 620 – and comparable to a 64-bit bus dedicated card.
	Upload Bandwidth (GB/s)	17.9 [+29%]	13.9	3 (PCIe3 x4)	The CPU<>GPU internal link seems to have 30% more bandwidth – naturally zero transfers are also supported. And a lot better than a dedicated card on PCIe3 x4 (4 lanes).
	Download Bandwidth (GB/s)	17.9 [+35%]	13.3	3 (PCIe3 x4)	Here again we see a good 35% bandwidth improvement.
CFL’s higher (stable) memory speed support improves bandwidth between 20-35% – which is likely behind most benchmark improvement in the compute algorithms above. However, that will only happen if high-speed DDR4 memory (3200 or faster) were to be used – an expensive proposition! eDRAM would greatly help here…

	Global (In-Page Random Access) Latency (ns)	179 [+1%]	178	223	No changes in global latencies in-page showing no memory sub-system improvements.
	Global (Full Range Random Access) Latency (ns)	268 [-19%]	332	244	Due to faster memory clock (even with slightly increased timings) full random access latencies fall by 20% (similar to bandwidth increase).
	Global (Sequential Access) Latency (ns)	126 [-5%]	132	76	Sequential access latencies do fall by a minor 5% as well though.
	Constant Memory (In-Page Random Access) Latency (ns)	181 [-6%]	192	92.5	Intel’s GPGPU don’t have dedicated constant memory thus we see similar performance to global memory.
	Shared Memory (In-Page Random Access) Latency (ns)	72 [-1%]	73	16.6	Shared memory latency is unchanged – and quite slow compared to architectures from competitors like the 1030.
	Texture (In-Page Random Access) Latency (ns)	138 [-9%]	151	220	Texture access latencies do seem to show a 9% improvement a surprising result.
	Texture (Full Range Random Access) Latency (ns)	227 [-16%]	270	242	Just as we’ve seen with global (full range access) latencies, we see the best improvement about 16% here.
	Texture (Sequential Access) Latency (ns)	45 [=]	45	71.9	With sequential access we see no improvement.
Anything to do with main memory access (aka “full random access”) does show a similar improvement to bandwidth increases, i.e. between 16-19% due to higher speed (but somewhat higher timings) main memory. All other access patterns show little to no improvements.

When using higher speed DDR4 memory – as we do here (3200 vs 2533) UHD630 shows a good improvement in both bandwidth and reduced latencies – but otherwise it performs just the same as the old HD520 – not a surprise really. At least you can see that your (expensive) memory investment does not go to waste – with memory bound algorithms showing good improvement.

SiSoftware Official Ranker Scores

Final Thoughts / Conclusions

For GPGPU workloads, UHD630 does not bring anything new – it performs similarly to the old HD520. But as CFL can use higher (stable) memory, bandwidth and latencies are improved (when using such higher speed memory) and thus most algorithms do show good improvements. Naturally as long as you can afford to provide such memory.

The surprising support for 1/2 ratio native FP64 support means 64-bit floating-point algorithms can run faster than on a typical low-end graphics card (as despite also supporting native FP64 the ratio is 1/32 vs. FP32 rate) so high accuracy workloads do work well on it. If loss of accuracy is OK (e.g. picture processing) native FP16 support at 2x rate makes such algorithms almost 2x faster and thus within the performance of a typical low-end graphics card (that either don’t support FP16 or their ratio is 1/64!).

As we touched in the introduction – this may not matter on desktop – but on mobile where most laptops/tablets use the integrated graphics any and all such improvements can make a big difference. While in the past the fast-improving EV cores became performance competitive with CPU cores (as there were only 2 ULV ones) – with CFL doubling number of CPU cores (4 vs. 2) it is likely that internal graphics (GPGPU) performance is now too low.

We’re sad that the GT3/GT4 versions are not common-place not to mention the L4/eDRAM which showed so much promise in the HSW days.

But Intel has recently revamped its GPU division and are committed to release dedicated (not just internal) graphics in a few years (2020?) which hopefully means we should see far more powerful GPUs from them soon.

Let’s hope they do see the light-of-day and are not cancelled like the “Phi” GPGPU accelerators (“Knights Landing”) which showed so much promise but somehow never made it outside data centres before sailing into the sunset…

Intel Core i7 8700K CofeeLake Review & Benchmarks – 2-channel DDR4 Cache & Memory Performance

By Adrian Silasi | 29th October 2018 - 1:13 pm |22nd November 2018 Reviews

Comments Off

What is “CofeeLake” CFL?

The 8th generation Intel Core architecture is code-named “CofeeLake” (CFL): unlike previous architectures, it is a minor stepping of the previous 7th generation “KabyLake” (KBL), itself a minor update of the 6th generation “SkyLake” (SKL). The server/workstation (SKL-X/KBL-X) CPU core saw new instruction set support (AVX512) as well as other improvements – these have not made the transition yet.

Possibly due limited competition (before AMD Ryzen launch), process issues (still at 14nm) and the disclosure of a whole host of hardware vulnerabilities (Spectre, Meltdown, etc.) which required microcode (firmware) updates – performance improvements have not been forthcoming. This is pretty much unprecedented – while some Core updates were only evolutionary we have not had complete stagnation before; in addition the built-in GPU core has also remained pretty much stagnant – we will investigate this in a subsequent article.

However, CFL does bring up a major change – and that is increased core counts both on desktop and mobile: on desktop we go from 4 to 6 cores (+50%) while on mobile (ULV) we go from 2 to 4 (+100%) within the same TDP envelope!

In this article we test CPU Cache and Memory performance; please see our other articles on:

Hardware Specifications

We are comparing the top-of-the-range Gen 8 Core i7 (8700K) with previous generation (6700K) and competing architectures with a view to upgrading to a mid-range high performance design.

CPU Specifications	Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Intel i7-6700K SkyLake	Comments
L1D / L1I Caches	6x 32kB 8-way / 6x 32kB 8-way	8x 32kB 8-way / 8x 64kB 8-way	10x 32kB 8-way / 10x 32kB 8-way	4x 32kB 8-way / 4x 32kB 8-way	No L1D/I changes, Ryzen’s L1I is twice as big.
L2 Caches	6x 256kB 4-way	8x 512kB 8-way	10x 1MB 16-way	4x 256kB 4-way	No L2 changes, Ryzen’s L2 is twice as big again.
L3 Caches	12MB 16-way	2x 8MB 16-way	2x 8MB 16-way	8MB 16-way	L3 has also increased with no of cores, still behind Ryzen’s dual 8MB L3 caches.
TLB 4kB pages	64 4-way / 64 8-way/ 1536 6-way	64 full-way 1536 8-way	64 4-way / 64 8-way / 1536 6-way	64 4-way / 64 8-way / 1536 6-way	No TLB changes.
TLB 2MB pages	8 full-way / 1536 6-way	64 full-way 1536 2-way	8 full-way / 1536 6-way	8 full-way / 1536 6-way	No TLB changes.
Memory Controller Speed (MHz)	1200-4400	1333-2667	1200-2700	1200-4000	The uncore (memory controller) runs at faster clock due to higher rated clock but not a lot in it.
Memory Data Speed (MHz)	3200	2667	3200	2533	CFL can easily run at 3200Mt/s while KBL/SKL were not as reliable. We could not get Ryzen past 2667 while it does support 2933.
Memory Channels / Width	2 / 128-bit	2 / 128-bit	2 / 128-bit	2 / 128-bit	All have 128-bit total channel width.
Memory Bandwidth (GB/s)	50	42	100	40	Bandwidth has naturally increased with memory clock speed but latencies are higher.
Uncore / Memory Controller Firmware	2.6.2			2.0.0.6	We’re on firmware 2.6.x vs. 2.0.x on old SKL/KBL.
Memory Timing (clocks)	16-16-16-36 6-52-25-12 2T	16-17-17-35 7-60-20-10 2T		16-18-18-36 5-54-21-10 2T	Timings are very much BIOS dependent and vary a lot.

Native Performance

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64 (1807), latest drivers. 2MB “large pages” were enabled and in use. Turbo / Boost was enabled on all configurations.

Spectre / Meltdown Windows Mitigations: all were enabled as per default (BTI enabled, RDCL/KVA enabled, PCID enabled).

Native Benchmarks		Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Intel i7-6700K SkyLake	Comments

	Total Inter-Core Bandwidth – Best (GB/s)	52.5 [-5%]	55.3	86	39.5	Despite just 2 less cores, CFL has only 5% less bandwidth than Ryzen 2.
	Total Inter-Core Bandwidth – Worst (GB/s)	15.5 [+144%]	6.35	25.7	16.1	In worst-case pairs on Ryzen2 must go across CCXes – unlike Intel’s CPUs – thus CFL can muster over 2x more bandwidth in this case.
CFL manages good bandwidth improvement over KBL/SKL – and due to unified design matching Ryzen2 in best case and beating it soundly in worst case.

	Inter-Unit Latency – Same Core (ns)	14.4 [+7%]	13.5	15	16	Surprisingly, Ryzen2 manages lower thread latency when sharing core.
	Inter-Unit Latency – Same Compute Unit (ns)	45 [+12%]	40	75	47	Within the same unit, Ryzen2 is again faster than CFL.
	Inter-Unit Latency – Different Compute Unit (ns)	–	115	–	–	Obviously going across CCXes is slow, about 3x slower which needs careful thread scheduling.
The multiple CCX design still presents some challenges to programmers requiring threads to be carefully scheduled – but we see Ryzen2 with lower latencies for both core and unit a surprising result as usually Intel’s caches are lower latency.

	Aggregated L1D Bandwidth (GB/s)	1630 [+59%]	854	2220	884	Intel’s wide data path L1 caches allow even old SKL to beat Ryzen2 with CFL enjoying 60% more bandwidth.
	Aggregated L2 Bandwidth (GB/s)	571 [-21%]	720	985	329	But Ryzen2’s L2 caches are not only twice as big but also very wide – CFL has 20% less bandwidth.
	Aggregated L3 Bandwidth (GB/s)	327 [-4%]	339	464	243	Ryzen’s 2 L3 caches also provide good bandwidth matching CFL’s unified L3 cache.
	Aggregated Memory (GB/s)	35.6 [+11%]	32.2	70	30.1	Running at 3200Mt’s obviously CFL enjoys higher bandwidth than Ryzen2 at 2667Mt’s but somehow the latter has better efficiency.
Nothing much has changed in CFL vs. old SKL thus while L1 caches are wide and thus fast – the L2, L3 are not as impressive and the memory controller while competitive it does not seem as efficient as Ryzen2 but is more stable at high data rates allowing for higher bandwidth.

	Data In-Page Random Latency (ns)	17.4 (4-11-20) [-73%]	63.4 (4-12-31)	25.5 (4-13-30)	20.4 (4-12-21)	While clock latencies have not changed w.s. old KBL/SKL, CFL enjoys lower latencies due to higher data rates. Ryzen2 has problems here.
	Data Full Random Latency (ns)	53.4 (4-11-42) [-30%]	76.2 (4-12-32)	74 (4-13-62)	63.9 (4-12-34)	Out-of-page clock latencies have increased but still overall lower. Ryzen2 has almost caught up here.
	Data Sequential Latency (ns)	3.8 (4-11-12) [+15%]	3.3 (4-6-7)	5.3 (4-12-12)	4.1 (4-12-13)	With sequential access, Ryzen2 is now faster as CFL’s clock latencies have not changed.
CFL is lucky here as even Ryzen2 still has high latencies in random accesses (either in-page or full range) but manages to be faster with sequential access. Intel will need to improve going forward as clock latencies while good have really not improved at all.

	Code In-Page Random Latency (ns)	8.7 (2-10-21) [-37%]	13.8 (4-9-24)	11.8 (4-14-25)	10.1 (2-10-21)	Code clock latencies also have not changed and again and while Ryzen2 performs a lot better, CFL (even old SKL) manage to be ~35% faster.
	Code Full Random Latency (ns)	59.8 (2-10-48) [-30%]	85.7 (4-14-49)	83.6 (4-15-74)	70.7 (2-11-46)	Out-of-page clock latencies also have not changed and here CFL is 20% faster over Ryzen2.
	Code Sequential Latency (ns)	4.5 (2-4-10) [-39%]	7.4 (4-12-20)	6.8 (4-7-11)	5 (2-4-9)	Ryzen2 is competitive but again CFL manages to be almost 40% faster.
CFL dominates here and enjoys 30-40% less latency over Ryzen2 but the latter has improved a lot in time.

	Memory Update Transactional (MTPS)	54 [+980%]	5	59	35	Finally all top-end Intel CPUs have HLE enabled and working and thus enjoy huge performance increase.
	Memory Update Record Only (MTPS)	38 [+730%]	4.58	59	24.8	Nothing much changes here.

Ryzen2 brings nice updates – good bandwidth increases to all caches L1D/L2/L3 and also well-needed latency reduction for data (and code) accesses. Yes, there is still work to be done to bring the latencies down further – but it may be just enough to beat Intel to 2nd place for a good while.

At the high-end, ThreadRipper2 will likely benefit most as it’s going against many-core SKL-X AVX512-enabled competitor which is a lot “tougher” than the normal SKL/KBL/CFL consumer versions.

SiSoftware Official Ranker Scores

Final Thoughts / Conclusions

CFL’s caches and memory (uncore) sub-systems are unchanged from SKL/KBL and thus provide no surprises, with rock-solid performance at 3200Mt/s with huge bandwidth (needed after all to feed 12 threads) but Ryzen2 has improved a lot over old AMD CPU designs.

With the continuous increase in cores/threads (8/12 in CFL-R) as with Ryzen1/2 but modest DDR4 speed increases (not to mention very high cost), the desktop platforms are likely to see diminishing returns due to core/thread data starvation while the extra cores just cannot be fed by the memory sub-systems. The L2 and L3 caches will need to be improved (widened, larger as with SKL-X) also the now defunct L4/eDRAM cache should re-emerge to mitigate these issues…

Intel Core i7 8700K CofeeLake Review & Benchmarks – CPU 6-core/12-thread Performance

By Adrian Silasi | 26th October 2018 - 4:07 pm |22nd November 2018 Reviews

Comments Off

What is “CofeeLake” CFL?

The 8th generation Intel Core architecture is code-named “CofeeLake” (CFL): unlike previous architectures, it is a minor stepping of the previous 7th generation “KabyLake” (KBL), itself a minor update of the 6th generation “SkyLake” (SKL). The server/workstation (SKL-X/KBL-X) CPU core saw new instruction set support (AVX512) as well as other improvements – these have not made the transition yet.

While this article is a bit late in the day considering the 8700K launched last year – we have now also reviewed the brand-new CofeeLake-R (Refresh) Core i9-9900K and it seems a good time to see what has changed performance-wise for the previous top-of-the-range CPU.

In this article we test CPU Core performance; please see our other articles on:

Hardware Specifications

We are comparing the top-of-the-range Gen 8 Core i7 (8700K) with previous generation (6700K) and competing architectures with a view to upgrading to a mid-range high performance design.

CPU Specifications	Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Intel i7-6700K SkyLake	Comments
Cores (CU) / Threads (SP)	6C/12T	8C / 16T	10C / 20T	4C / 8T	We have 50% more cores compared to SKL/KBL but still not as much as Ryzen/2 with 8 cores.
Speed (Min / Max / Turbo)	0.8-3.7-4.7GHz (8x-37x-47x)	2.2-3.7-4.2GHz (22x-37x-42x)	1.2-3.3-4.3 (12x-33x-43x)	0.8-4.0-4.2GHz (8x-40x-42x)	Single-core Turbo has increased close to 5GHz (reserved for Special Edition 8086K) way above SKL/KBL and Ryzen.
Power (TDP)	95W (131)	105W (135)	140W (308)	91W (100)	TDP has only increased by 4% and is still below Ryzen though Turbo is comparable.
L1D / L1I Caches	6x 32kB 8-way / 6x 32kB 8-way	8x 32kB 8-way / 8x 64kB 8-way	10x 32kB 8-way / 10x 32kB 8-way	4x 32kB 8-way / 4x 32kB 8-way	No change in L1 caches. Just more of them.
L2 Caches	6x 256kB 8-way	8x 512kB 8-way	10x 1MB 8-way	4x 256kB 8-way	No change in L2 caches. Just more of them.
L3 Caches	12MB 16-way	2x 8MB 16-way	13.75MB 11-way	8MB 16-way	L3 has also increased by 50% in line with cores, but still below Ryzen’s 16MB.
Microcode/Firmware	MU069E0A-96	MU8F0802-04	MU065504-49	MU065E03-C2	We have a new model and somewhat newer microcode.

Native Performance

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64 (1807), latest drivers. 2MB “large pages” were enabled and in use. Turbo / Boost was enabled on all configurations.

Spectre / Meltdown Windows Mitigations: all were enabled as per default (BTI enabled, RDCL/KVA enabled, PCID enabled).

Native Benchmarks		Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Intel i7-6700K SkyLake	Comments

	Native Dhrystone Integer (GIPS)	291 [-13%]	334	485	190	In the old Drystone integer workload, CFL is still 13% slower than Ryzen 2 depite the huge lead over SKL.
	Native Dhrystone Long (GIPS)	296 [-12%]	335	485	192	With a 64-bit integer workload – nothing much changes.
	Native FP32 (Float) Whetstone (GFLOPS)	170 [-14%]	198	262	105	Switching to floating-point, CFL is still 14% slower in the old Whetstone also a micro-benchmark.
	Native FP64 (Double) Whetstone (GFLOPS)	143 [-15%]	169	223	89	With FP64 nothing much changes.
From integer workloads in Dhyrstone to floating-point workloads in Whestone, CFL is still 12-15% slower than Ryzen 2 with its 2 more cores (8 vs. 2), but much faster than the old SKL with 4 cores. We begin to see now why Intel is adding more cores in CFL-R.

	Native Integer (Int32) Multi-Media (Mpix/s)	741 [+29%]	574	1590 (AVX512)	474	In this vectorised AVX2 integer test we see CFL beating Ryzen by ~30% despite less cores.
	Native Long (Int64) Multi-Media (Mpix/s)	305 [+63%]	187	581 (AVX512)	194	With a 64-bit AVX2 integer vectorised workload, CFL is now 63% faster.
	Native Quad-Int (Int128) Multi-Media (Mpix/s)	4.9 [-16%]	5.8	7.6	3	This is a tough test using Long integers to emulate Int128 without SIMD: Ryzen 2 thus wins this one with CFL slower by 16%.
	Native Float/FP32 Multi-Media (Mpix/s)	678 [+14%]	596	1760 (AVX512)	446	In this floating-point AVX/FMA vectorised test, CFL is again 14% faster.
	Native Double/FP64 Multi-Media (Mpix/s)	402 [+20%]	335	533 (AVX512)	268	Switching to FP64 SIMD code, CFL is again 20% faster.
	Native Quad-Float/FP128 Multi-Media (Mpix/s)	16.7 [+7%]	15.6	40.3 (AVX512)	11	In this heavy algorithm using FP64 to mantissa extend FP128 but not vectorised – CFL is just 7% faster but does win.
In vectorised SIMD code we see the power of Intel’s SIMD units that can execute 256-bit instructions in one go; CFL soundly beats Ryzen2 despite fewer cores (7-60%). SKL-X shows that AVX512 brings further gains and is a pity CFL still does not support them.

	Crypto AES-256 (GB/s)	17.8 [+11%]	16.1	23	15	With AES HWA support all CPUs are memory bandwidth bound; unfortunately Ryzen 2 is at 2667 vs CFL/SKL-X at 3200 which means CFL is 11% faster.
	Crypto AES-128 (GB/s)	17.8 [+11%]	16.1	23	15	What we saw with AES-256 just repeats with AES-128.
	Crypto SHA2-256 (GB/s)	9 [-51%]	18.6	26 (AVX512)	5.9	With SHA HWA Ryzen2 similarly powers through hashing tests leaving Intel in the dust; CFL is thus 50% slower.
	Crypto SHA1 (GB/s)	17.3 [-9%]	19.3	38 (AVX512)	11.2	Ryzen also accelerates the soon-to-be-defunct SHA1 but the algorithm is less compute heavy thus CFL is only 9% slower.
	Crypto SHA2-512 (GB/s)	6.65 [+77%]	3.77	21 (AVX512)	4.4	SHA2-512 is not accelerated by SHA HWA, allowing CFL to use its SIMD units and be 77% faster.
AES HWA is memory bound and here CFL comfortably works with 3200Mt/s memory thus is faster than Ryzen 2 with 2667Mt/s memory (our sample); likely both would score similarly at 3200Mt/s. Ryzen 2 SHA HWA allows it to easily beat all other CPUs but only in SHA1/SHA256 – in others the CFL SIMD units win the day.

	Black-Scholes float/FP32 (MOPT/s)	207 [-19%]	257	309	128	In this non-vectorised test CFL cannot match Ryzen 2 and is ~20% slower.
	Black-Scholes double/FP64 (MOPT/s)	180 [-18%]	219	277	113	Switching to FP64 code, nothing much changes, Ryzen 2 is still faster.
	Binomial float/FP32 (kOPT/s)	47 [-56%]	107	70.5	29.3	Binomial uses thread shared data thus stresses the cache & memory system; Ryzen 2 does very well here with CFL almost 60% slower.
	Binomial double/FP64 (kOPT/s)	44.2 [-27%]	60.6	68	27.3	With FP64 code Ryzen2’s lead diminishes, CFL is “only” 27% slower.
	Monte-Carlo float/FP32 (kOPT/s)	41.6 [-23%]	54.2	63	25.7	Monte-Carlo also uses thread shared data but read-only thus reducing modify pressure on the caches; Ryzen 2 also wins this one, CFL is 23% slower.
	Monte-Carlo double/FP64 (kOPT/s)	32.9 [-20%]	41	50.5	20.3	Switching to FP64 nothing much changes, CFL is still 20% slower.
Without SIMD support, CFL loses to Ryzen 2 as we saw with Dhrystone/Whetstone – between 20 and 50%. As we noted before, Intel will still need to add more cores in order to beat Ryzen 2. Still big improvement over the old SKL/KBL as expected.

	SGEMM (GFLOPS) float/FP32	385 [+28%]	300	413 (AVX512)	268	In this tough vectorised AVX2/FMA algorithm CFL is ~30% faster.
	DGEMM (GFLOPS) double/FP64	135 [+13%]	119	212 (AVX512)	130	With FP64 vectorised code, CFL’s lead reduces to 13% over Ryzen 2.
	SFFT (GFLOPS) float/FP32	24 [167%]	9	28.6 (AVX512)	16.1	FFT is also heavily vectorised (x4 AVX/FMA) but stresses the memory sub-system more; here CFL is over 2.5x faster
	DFFT (GFLOPS) double/FP64	11.9 [+51%]	7.92	14.6 (AVX512)	7.2	With FP64 code, CFL’s lead reduces to ~50%.
	SNBODY (GFLOPS) float/FP32	411 [+47%]	280	638 (AVX512)	271	N-Body simulation is vectorised but many memory accesses to shared data but CFL remains ~50% faster.
	DNBODY (GFLOPS) double/FP64	127 [+13%]	113	195 (AVX512)	79	With FP64 code CFL’s lead reduces to 13% over Ryzen 2.
With highly vectorised SIMD code CFL performs well soundly beating Ryzen 2 between 13-167% as well as significantly improving over older SKL/KBL. As long as SIMD code is used Intel has little to fear.

	Blur (3×3) Filter (MPix/s)	1700 [+39%]	1220	4540 (AVX512)	1090	In this vectorised integer AVX2 workload CFL enjoys a ~40% lead over Ryzen 2.
	Sharpen (5×5) Filter (MPix/s)	675 [+25%]	542	1790 (AVX512)	433	Same algorithm but more shared data reduces the lead to 25% still significant.
	Motion-Blur (7×7) Filter (MPix/s)	362 [+19%]	303	940 (AVX512)	233	Again same algorithm but even more data shared reduces the lead to 20%.
	Edge Detection (2*5×5) Sobel Filter (MPix/s)	589 [+30%]	453	1520 (AVX512)	381	Different algorithm but still AVX2 vectorised workload means CFL is 30% faster than Ryzen 2.
	Noise Removal (5×5) Median Filter (MPix/s)	57.8 [-17%]	69.7	223 (AVX512)	37.6	Still AVX2 vectorised code but CFL stumbles a bit here – it’s 17% slower than Ryzen 2.
	Oil Painting Quantise Filter (MPix/s)	31.8 [+29%]	24.6	70.8 (AVX512)	20	Again we see CFL ~30% faster.
	Diffusion Randomise (XorShift) Filter (MPix/s)	3480 [+140%]	1450	3570 (AVX512)	2300	CFL (like all Intel CPUs) does very well here – it’s a huge 140% faster.
	Marbling Perlin Noise 2D Filter (MPix/s)	448 [+84%]	243	909 (AVX512)	283	In this final test, CFL is almost 2x faster than Ryzen 2.

The addition of 2 more cores brings big performance gains (not to mention the higher Turbo clock) over the old SKL/KBL which is pretty impressive considering TDP has stayed the same. With SIMD code (AVX/AVX2/FMA3) CFL has no problem beating Ryzen 2 by a pretty large margin (up to 2x faster) – but any algorithm not vectorised allows Ryzen 2 to win – though not by much 12-20%.

Streaming tests likely benefit from the higher supported memory frequencies that while in theory could be used on the older SKL/KBL (memory overclock) they were not supported officially nor stable in all cases. We shall test memory performance in a forthcoming article.

Software VM (.Net/Java) Performance

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64 (1807), latest drivers. 2MB “large pages” were enabled and in use. Turbo / Boost was enabled on all configurations.

Spectre / Meltdown Windows Mitigations: all were enabled as per default (BTI enabled, RDCL/KVA enabled, PCID enabled).

VM Benchmarks		Intel i7-8700K CofeeLake	AMD Ryzen2 2700X Pinnacle Ridge	Intel i9-7900X SkyLake-X	Intel i7-6700K SkyLake	Comments

	.Net Dhrystone Integer (GIPS)	41 [-33%]	61	52	28	.Net CLR integer performance starts off well over old SKL but still 33% slower than Ryzen 2.
	.Net Dhrystone Long (GIPS)	41 [-32%]	60	54	27	With 64-bit integers nothing much changes.
	.Net Whetstone float/FP32 (GFLOPS)	78 [-24%]	102	107	49	Floating-Point CLR performance does not change much, CFL is still 25% slower than Ryzen despite big gain over old SKL.
	.Net Whetstone double/FP64 (GFLOPS)	95 [-19%]	117	137	62	FP64 performance is similar to FP32.
Ryzen 2 performs exceedingly well in .Net workloads – soundly beating all Intel CPUs, with CFL between 20-33% slower. More cores will be needed for parity with Ryzen 2, but at least CFL improves a lot over SKL/KBL.

	.Net Integer Vectorised/Multi-Media (MPix/s)	93.5 [-16%]	111	144	57	Just as we saw with Dhrystone, this integer workload sees CFL improve greatly over SKL but Ryzen 2 is still faster.
	.Net Long Vectorised/Multi-Media (MPix/s)	93.1 [-15%]	109	143	57	With 64-bit integer workload nothing much changes.
	.Net Float/FP32 Vectorised/Multi-Media (MPix/s)	361 [-8%]	392	585	228	Here we make use of RyuJit’s support for SIMD vectors thus running AVX/FMA code but CFL is still 8% slower than Ryzen 2.
	.Net Double/FP64 Vectorised/Multi-Media (MPix/s)	198 [-9%]	217	314	128	Switching to FP64 SIMD vector code – still running AVX/FMA – CFL is still slower.
We see a similar improvement for CFL but again not enough to beat Ryzen 2; even using RyuJit’s vectorised support CFL cannot beat it – just reduce the loss to 8-9%.

	Java Dhrystone Integer (GIPS)	557 [-3%]	573	877	352	Java JVM performance is almost neck-and-neck with Ryzen 2 despite 2 less cores.
	Java Dhrystone Long (GIPS)	488 [-12%]	553	772	308	With 64-bit integers, CFL does fall behind Ryzen2 by 12%.
	Java Whetstone float/FP32 (GFLOPS)	101 [-23%]	131	156	62	Floating-point JVM performance is worse though, CFL is now 23% slower.
	Java Whetstone double/FP64 (GFLOPS)	103 [-26%]	139	160	64	With 64-bit precision nothing much changes.
While CFL improves markedly over the old SKL and almost ties with Ryzen 2 in integer workloads, it does fall behind in floating-point by a good amount.

	Java Integer Vectorised/Multi-Media (MPix/s)	100 [-12%]	113	140	63	Without SIMD acceleration we see the usual delta (around 12%) with integer workload.
	Java Long Vectorised/Multi-Media (MPix/s)	89 [-12%]	101	152	62	Nothing changes when changing to 64-bit integer workloads.
	Java Float/FP32 Vectorised/Multi-Media (MPix/s)	64 [-34%]	97	98	41	With floating-point non-SIMD accelerated we see a bigger delta of about 30% less vs. Ryzen 2.
	Java Double/FP64 Vectorised/Multi-Media (MPix/s)	64 [-29%]	90	99	41	With 64-bit floatint-point precision nothing much changes.
With compute heavy vectorised code but not SIMD accelerated, CFL cannot keep up with Ryzen 2 and the difference increases to about 30% less. Intel really needs to get Oracle to add SIMD extensions similar to .Net’s new CLR.

Ryzen dominates .Net and Java benchmarks – Intel will need more cores in order to compete; while the additional 2 cores helped a lot, it is not enough!

SiSoftware Official Ranker Scores

Final Thoughts / Conclusions

Due to Core improvement stagnation (4 on desktop, 2 on mobile), Intel had no choice really but increase core counts in light of new competition from AMD with Ryzen/2 with twice as many cores (8) and SMT (Hyper-Threading) as well! While KBL had increased base/Turbo cores appreciably over SKL within the same power envelope, CFL had to add more cores in order to compete.

With 50% more cores (6) CFL performs much better than the older SKL/KBL as expected but that is not enough in non-vectorised loads; Ryzen 2 with 2 more cores is still faster, not by much (12-20%) but still faster. Once vectorised code is used, the power of Intel’s SIMD units shows – with CFL soundly beating Ryzen despite not supporting AVX512 nor SHA HWA which is a pity. AMD still has work to do with Ryzen if it wants to be competitive in vectorised workloads (integer or floating-point).

We now see why CFL-R (CofeeLake Refresh) will add even more cores (8C/16T with 9900K which we review in a subsequent article) – it is the only way to beat Ryzen 2 in all workloads. In effect AMD’s has reached parity performance with Intel in all but SIMD workloads – a great achievement!

Unfortunately (unlike AMD’s AM4 Ryzen) CFL does require new chipset/boards (series 300) which makes it an expensive upgrade for SKL/KBL owners; otherwise it would have been a pretty no-brainer upgrade for those needing more compute power. While the new platform does bring some improvements (USB 3.1 Gen 2 aka 10GB/s, more PCIe lanes, integrated 802.11ac WiFi – at least on mobile) it’s nothing over the competition.

Roll on CofeeLake Refresh and the new CPUs: they are sorely needed…

SiSoftware Sandra Titanium (2018) SP2b Update

By Adrian Silasi | 24th October 2018 - 12:58 am |30th October 2018 Releases, Updates

Comments Off

We are pleased to announce SP2b (version 28.31) for Sandra Titanium (2018) with updated hardware and software support:

Sandra Titanium (2018) Press Release

CPU Benchmarks:
- Multi-Media, .Net Multi-Media, Java Multi-Media: optimised fractal picture size selection for better scaling. Crash fix upon loading large fractals.
- Image Processing: optimised picture size selection for better scaling.
GPGPU Benchmarks:
- Processing (CUDA, OpenCL, DirectX): optimised fractal picture size selection for better scaling. Crash fix upon loading large fractals.
- Image Processing (CUDA, OpenCL, DirectX): optimised picture size selection for better scaling.
GPGPU: improved clock speed reporting (base/boost).

Reviews using Sandra 2018 SP2b

Commercial version customers can download the free updates from their software distributor; Lite users please download from your favourite download site.

Download Sandra Lite

SiSoftware Sandra Titanium (2018) SP2 Update

By Adrian Silasi | 25th September 2018 - 11:12 pm |29th October 2018 Releases, Updates

Comments Off

We are pleased to announce SP2 (version 28.28) for Sandra Titanium (2018) with updated hardware and software support:

Sandra Titanium (2018) Press Release

GPGPU:
- nVidia Series 2000 (Turing) GPGPU support (still on SDK 9.2 for broad compatibility)
- Fixes workgroup sizes for Scientific benchmarks
CPU:
- Support for Intel 9000-Series CPU
- Crypto AES benchmark VAES 512-bit and 256-bit support

Product Reviews

Commercial version customers can download the free updates from their software distributor; Lite users please download from your favourite download site.

Download Sandra Lite

SiSoftware Sandra Titanium (2018) SP1b Update

By Adrian Silasi | 13th August 2018 - 5:32 pm |29th October 2018 Releases, Updates

Comments Off

We are pleased to announce SP1b (version 28.26) for Sandra Titanium (2018) with updated hardware and software support:

Sandra Titanium (2018) Press Release

GPGPU: Updated nVidia CUDA 9.2 SDK with FP16 support for all benchmarks
- nVidia Volta arch GPGPU support
CPU: Support update for AMD ThreadRipper v2
CPU: Support update for Intel v9 CPUs

Product Reviews

Commercial version customers can download the free updates from their software distributor; Lite users please download from your favourite download site.

Download Sandra Lite

nVidia Titan V: Volta GPGPU performance in CUDA and OpenCL

By Adrian Silasi | 31st July 2018 - 4:31 pm |31st July 2018 Reviews

Comments Off

What is “Titan V”?

It is the latest high-end “pro-sumer” card from nVidia with the next-generation “Volta” architecture, the next generation to the current “Pascal” architecture on the Series 10 cards. Based on the top-end 100 chipset (not lower 102 or 104) it boasts full speed FP64/FP16 performance as well as brand-new “tensor cores” (matrix multipliers) for scientific and deep-learning workloads. It also comes with on-chip HBM2 (high-bandwidth) memory unlike more traditional GDDRX stand-alone memory.

For this reason the price is also far higher than previous Titan X/XP cards but considering the features/performance are more akin to “Tesla” series it would still be worth it depending on workload.

While using the additional cores provided in FP64/FP16 workloads is automatic – save usual code optimisations – tensor cores support requires custom code and existing libraries and apps need to be updated to make use of them. It is unknown at this time if consumer cards based on “Volta” will also include them. As they support FP16 precision only, not workloads may be able to use them – but DL (deep learning) and AI (artificial intelligence) are generally fine using lower precision thus for such tasks it is ideal.

See these other articles on Titan performance:

nVidia Titan X : Pascal GPGPU performance in CUDA and OpenCL

Hardware Specifications

We are comparing the top-of-the-range Titan V with previous generation Titans and competing architectures with a view to upgrading to a mid-range high performance design.

GPGPU Specifications	nVidia Titan V	nVidia Titan X (P)	nVidia 980 GTX (M2)	Comments
Arch Chipset	Volta VP100 (7.0)	Pascal GP102 (6.1)	Maxwell 2 GM204 (5.2)	The V is the only one using the top-end 100 chip not 102 or 104 lower-end versions
Cores (CU) / Threads (SP)	80 / 5120	28 / 3584	16 / 2048	The V boasts 80 CU units but these contain 64 FP32 units only not 128 like lower-end chips thus equivalent with 40.
FP32 / FP64 / Tensor Cores	5120 / 2560 / 640	3584 / 112 / no	2048 / 64 / no	Titan V is the only one with tensor cores and also huge amount of FP64 cores that Titan X simply cannot match; it also has full speed FP16 support.
Speed (Min-Turbo)	1.2GHz (135-1.455)	1.531GHz (139-1910)	1.126GHz (135-1.215)	Slightly lower clocked than the X it will will make up for it with sheer CU units.
Power (TDP)	300W	250W (125-300)	180W (120-225)	TDP increases by 50W but it is not unexpected considering the additional units.
ROP / TMU	96 / 320	96 / 224	64 / 128	Not a “gaming card” but while ROPs stay the same the number of TMUs has increased – likely required for compute tasks using textures.
Global Memory	12GB HBM2 850Mhz 3072-bit	12GB GDDR5X 10Gbps 384-bit	4GB GDDR5 7Gbps 256-bit	Memory size stays the same at 12GB but now uses on-chip HBM2 for much higher bandwidth
Memory Bandwidth (GB/s)	652	512	224	In addition to the modest bandwidth increase, latencies are also meant to have decreased by a good amount.
L2 Cache	4.5MB	3MB	2MB	L2 cache has gone up by about 50% to feed all the cores.
FP64/double ratio	1/2	1/32	1/32	For FP64 workloads the V has huge advantage as consumer and previous Titan X had far less FP64 units.
FP16/half ratio	2x	1/64	n/a	The V has an even bigger advantage here with over 128x more units for FP16 tasks like DL and AI.

Processing Performance

We are testing both CUDA native as well as OpenCL performance using the latest SDK / libraries / drivers.

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64, latest nVidia drivers 398.36, CUDA 9.2, OpenCL 1.2. Turbo / Boost was enabled on all configurations.

Processing Benchmarks		nVidia Titan V CUDA/OpenCL	nVidia Titan X CUDA/OpenCL	nVidia GTX 980 CUDA/OpenCL	Comments

	Mandel FP32/Single (Mpix/s)	22,400 [+25%] / 20,000	17,870 / 16,000	7,000 / 6,100	Right off the bat, the V is just 25% faster than the X some optimisations may be required.
	Mandel FP16/Half (Mpix/s)	33,300 [135x] / n/a	245 / n/a	n/a	For FP16 workloads the V shows its power: it is an astonishing 135 times (times not %) faster than the X.
	Mandel FP64/Double (Mpix/s)	11,000 [+16.7x] / 11,000	661 / 672	259 / 265	For FP64 precision workloads the V shines again, it is 16 times faster than the X.
	Mandel FP128/Quad (Mpix/s)	458 [+17.7x] / 455	25 / 24	10.8 / 10.7	With emulated FP128 precision the V is again 17 times faster.
As expected FP64 and FP16 performance is much improved on Titan V, with FP64 over 16x times faster than the X; FP16 performance is over 50% faster than FP32 performance making it almost 2x faster than Titan X. For workloads that need it, the performance of Titan V is stellar.

	Crypto AES-256 (GB/s)	71 [+79%] / 87	40 / 38	16 / 16	Titan V is almost 80% faster than the X here a significant improvement.
	Crypto AES-128 (GB/s)	91 [+75%] / 116	52 / 51	23 / 21	Not a lot changes here, with the V still 7% faster than the X.

	Crypto SHA2-256 (GB/s)	253 [+89%] / 252	134 / 142	58 / 59	In this integer workload, Titan V is almost 2x faster than the X.
	Crypto SHA1 (GB/s)	130 [+21%] / 134	107 / 114	50 / 54	SHA1 is mysteriously slower than SHA256 and here the V is just 21% faster.
	Crypto SHA2-512 (GB/s)	173 [+2.4x] / 176	72 / 42	32 / 24	With 64-bit integer workload, Titan V shines again – it is almost 2.5x (times) faster than the X!
Historically, nVidia cards have not been tuned for integer workloads, but Titan V is almost 2x faster in 32-bit hashing and almost 3x faster in 64-bit hashing than the older X. For algorithms that use integer computation this can be quite significant.

	Black-Scholes float/FP32 (MOPT/s)	18,460 [+61%] / 18,870	11,480 / 11,470	5,280 / 5,280	Titan V manages to be 60% faster in this FP32 financial workload.
	Black-Scholes double/FP64 (MOPT/s)	8,400 [+6.1x] / 9,200	1,370 / 1,300	547 / 511	Switching to FP64 code, the V is over 6x (times) faster than the X.
	Binomial float/FP32 (kOPT/s)	4,180 [+81%] / 4,190	2,240 / 2,240	1,200 / 1,140	Binomial uses thread shared data thus stresses the SMX’s memory system: but the V is 80% faster than the X.
	Binomial double/FP64 (kOPT/s)	2,000 [+15.5x] / 2,000	129 / 133	51 / 51	With FP64 code the V is much faster – 15x (times) faster!
	Monte-Carlo float/FP32 (kOPT/s)	12,550 [+2.35x] / 12,610	5,350 / 5,150	2,140 / 2,000	Monte-Carlo also uses thread shared data but read-only thus reducing modify pressure – here the V is over 2x faster than the X and that is FP32 code!
	Monte-Carlo double/FP64 (kOPT/s)	4,440 [+15.1x] / 4,100	294 / 267	118 / 106	Switching to FP64 the V is again over 15x (times) faster!
For financial workloads, the Titan V is significantly faster, almost twice as fast as Titan X on FP32 but over 15x (times) faster on FP64 workloads. If time is money, then this can be money well-spent!

	SGEMM (GFLOPS) float/FP32	9,860 [+57%] / 10,350	6,280 / 6,600	2,550 / 2,550	Without using the new “tensor cores”, Titan V is about 60% faster than the X.
	DGEMM (GFLOPS) double/FP64	3,830 [+11.4x] / 3,920	335 / 332	130 / 129	With FP64 precision, the V crushes the X again it is 11x (times) faster.
	SFFT (GFLOPS) float/FP32	605 [+2.5x] / 391	242 / 227	148 / 136	FFT allows the V to do even better – no doubt due to HBM2 memory.
	DFFT (GFLOPS) double/FP64	280 [+35%] / 245	207 / 191	89 / 82	We may need some optimisations here, otherwise the V is just 35% faster.
	SNBODY (GFLOPS) float/FP32	6,390 [+15%] / 4,630	5,600 / 4,870	2,100 / 2,000	N-Body simulation also needs some optimisations as the V is just 15% faster.
	DNBODY (GFLOPS) double/FP64	4,270 [+15.5x] / 4,200	275 / 275	82 / 81	With FP64 precision, the V again crushes the X – it is 15x faster.
The scientific scores are a bit more mixed – GEMM will require code paths to take advantage of the new “tensor cores” and some optimisations may be required – otherwise FP64 code simply flies on Titan V.

	Blur (3×3) Filter single/FP32 (MPix/s)	26,790 [50%] / 26,660	17,860 / 13,680	7,310 / 5,530	In this 3×3 convolution algorithm, Titan V is 50% faster than the X. Convolution is also used in neural nets (CNN) thus performance here counts.
	Blur (3×3) Filter half/FP16 (MPix/s)	29,200 [+18.6x]	1,570	n/a	With FP16 precision, Titan V shines it is 18x (times faster than X) but 12% faster than FP32.
	Sharpen (5×5) Filter single/FP32 (MPix/s)	9,295 [+94%] / 6,750	4,800 / 3,460	1,870 / 1,380	Same algorithm but more shared data allows the V to be almost 2x faster than the X.
	Sharpen (5×5) Filter half/FP16 (MPix/s)	14,900 [24.4x]	609	n/a	With FP16 Titan V is almost 25x (times) faster than X and also 60% faster than Fp32.
	Motion-Blur (7×7) Filter single/FP32 (MPix/s)	9,428 [+2x] / 7,260	4,830 / 3,620	1,910 / 1,440	Again same algorithm but even more data shared the V is 2x faster than the X.
	Motion-Blur (7×7) Filter half/FP16 (MPix/s)	14,790 [+45x]	325	n/a	With FP16 the V is now45x (times) faster than the X showing the usefulness of FP16 support.
	Edge Detection (2*5×5) Sobel Filter single/FP32 (MPix/s)	9,079 [1.92x] / 7,380	4,740 / 3450	1,860 / 1,370	Still convolution but with 2 filters – Titan V is almost 2x faster again.
	Edge Detection (2*5×5) Sobel Filter half/FP16 (MPix/s)	13,740 [+44x]	309	n/a	Just as we seen above, the V is an astonishing 44x (times) faster than the X, and also ~20% faster than FP32 code.
	Noise Removal (5×5) Median Filter single/FP32 (MPix/s)	111 [+3x] / 66	36 / 55	20 / 25	Different algorithm but here the V is even faster, 3x faster than the X!
	Noise Removal (5×5) Median Filter half/FP16 (MPix/s)	206 [+2.89x]	71	n/a	With FP16 the V is “only” 3x faster than the X but also 2x faster than FP32 code-path again a big gain for FP16 processing
	Oil Painting Quantise Filter single/FP32 (MPix/s)	157 [+10x] / 24	15 / 15	12 / 11	Without major processing, this filter flies on the V – it is 10x faster than the X.
	Oil Painting Quantise Filter half/FP16 (MPix/s)	215 [+4x]	50		FP16 precision is “just” 4x faster but it is also ~40% faster than FP32.
	Diffusion Randomise (XorShift) Filter single/FP32 (MPix/s)	24,370 / 22,780 [+25%]	19,480 / 14,000	7,600 / 6,640	This algorithm is 64-bit integer heavy and here Titan V is 25% faster than the X.
	Diffusion Randomise (XorShift) Filter half/FP16 (MPix/s)	24,180 [+4x]	6,090		FP16 does not help a lot here, but still the V is 4x faster than the X.
	Marbling Perlin Noise 2D Filter single/FP32 (MPix/s)	846 [+3x] / 874	288 / 635	210 / 308	One of the most complex and largest filters, Titan V does very well here, it is 3x faster than the X.
	Marbling Perlin Noise 2D Filter half/FP16 (MPix/s)	1,712 [+3.7x]	461	n/a	Switching to FP16, the V is almost 4x (times) faster than the X and over 2x faster than FP32 code.
For image processing, Titan V brings big performance increases from 50% to 4x (times) faster than Titan X a big upgrade. If you are willing to drop to FP16 precision, then it is an extra 50% to 2x faster again – while naturally FP16 is not really usable on the X. With potential 8x times better performance Titan V powers through image processing tasks.

Memory Performance

We are testing both CUDA native as well as OpenCL performance using the latest SDK / libraries / drivers.

Results Interpretation: For bandwidth tests (MB/s, etc.) high values mean better performance, for latency tests (ns, etc.) low values mean better performance.

Environment: Windows 10 x64, latest nVidia drivers 398.36, CUDA 9.2, OpenCL 1.2. Turbo / Boost was enabled on all configurations.

HBM2 does seem to increase latencies slightly by about 10% but for sequential accesses Titan V does perform a lot better than the X with 20-40% lower latencies, likely due to the the new architecture. Thus code using coalesce memory accesses will perform faster but for code using random access pattern over large data sets

Memory Benchmarks		nVidia Titan V CUDA/OpenCL	nVidia Titan X CUDA/OpenCL	nVidia GTX 980 CUDA/OpenCL	Comments

	Internal Memory Bandwidth (GB/s)	536 [+51%] / 530	356 / 354	145 / 144	HBM2 brings about 50% more raw bandwidth to feed all the extra compute cores, a significant upgrade.
	Upload Bandwidth (GB/s)	11.47 / 11,4	11.4 / 9	12.1 / 12	Still using PCIe3 x16 there is no change in upload bandwidth. Roll on PCIe4!
	Download Bandwidth (GB/s)	12.3 / 12.3	12.2 / 8.9	11.5 / 12.2	Again no significant difference but we were not expecting any.
Titan V’s HBM2 brings 50% more memory bandwidth but as it still uses the PCIe3 x16 connection there is no change to host upload/download bandwidth which may be a bit of a bottleneck trying to keep all those cores fed with data. Even more streaming load/save is required and code will need to be optimised to use all that processing power

	Global (In-Page Random Access) Latency (ns)	180 [-10%] / 187	201 / 230	230	From the start we see global latency accesses reduced by 10%, not a lot but will help.
	Global (Full Range Random Access) Latency (ns)	311 [+9%] / 317	286 / 311	306	Full range random accesses do seem to be 9% slower which may be due to the architecture.
	Global (Sequential Access) Latency (ns)	53 [-40%] / 57	89 / 121	97	However, sequential accesses seem to have dropped a huge 40% likely due to better prefetchers on the Titan V.
	Constant Memory (In-Page Random Access) Latency (ns)	75 [-36%] / 76	117 / 174	126	Constant memory latencies also seem to have dropped by almost 40% a great result.
	Shared Memory (In-Page Random Access) Latency (ns)	18 / 85	18 / 53	21	No significant change in shared memory latencies.
	Texture (In-Page Random Access) Latency (ns)	212 [+9%] / 279	195 / 196	208	Texture access latencies seem to have increased by 9%
	Texture (Full Range Random Access) Latency (ns)	344 [+22%] / 313	282 / 278	308	As we’ve seen with global memory, we see increased latencies here by about 20%.
	Texture (Sequential Access) Latency (ns)	88 / 163	87 /123	102	With sequential access there is no appreciable delta in latencies.
HBM2 does seem to increase latencies slightly by about 10% but for sequential accesses Titan V does perform a lot better than the X with 20-40% lower latencies, likely due to the the new architecture. Thus code using coalesce memory accesses will perform faster but for code using random access pattern over large data sets

We see L1 cache effects between 64-128kB tallying with an L1D of 96kB – 4x more than what we’ve seen on Titan X (at 16kB). The other inflexion is at 4MB – matching the 4.5MB L2 cache size – which is 50% more than what we saw on Titan X (at 3MB).

As with global memory we see the same L1D (64kB) and L2 (4.5MB) cache affects with similar latencies. Both are significant upgrades over Titan X’ caches.

Titan V’s memory performance does not disappoint – HBM2 obviously brings large bandwidth increase – latency depends on access pattern, when prefetchers can engage they are much lowers but in random accesses out-of-page they are a big higher but nothing significant. We’re also limited by the PCIe3 bus for transfers which requires judicious overlap of memory transfers and compute to keep the cores busy.

SiSoftware Official Ranker Scores

Final Thoughts / Conclusions

“Volta” architecture does bring good improvements in FP32 performance which we hope to see soon in consumer (Series 11?) graphics cards – as well as lower-end Titan cards.

But here (on Titan V) we have the top-end chip with full-power FP64 and FP16 units more akin to Tesla which simply power through any and all algorithms you can throw at them. This is really the “Titan” you were looking for and upgrading from the previous Titan X (Pascal) is a huge upgrade admittedly for quite a bit more money.

If you have workloads that requires double/FP64 precision – Titan V is 15-16x times faster than Titan X – thus great value for money. If code can make do with FP16 precision then you can gain up to 2x extra performance again – as well as save storage for large data-sets – again Titan X cannot cut it here running at 1/64 rate.

We have not yet shown tensor core performance which is an additional reason for choosing such a card – if you have code that can make use of them you can gain an extra 16x (times) performance that really puts Titan V heads and shoulders over the Titan X.

All in all Titan V is a compelling upgrade if you need more power than Titan X and are (or thinking of) using multiple cards – there is simply no point. One Titan V can replace 4 or more Titan X cards on FP64 or FP16 workloads and that is before you make any optimisations. Obviously you are still “stuck” with 12GB memory and PCIe bus for transfers but with judicious optimisations this should not impact performance significantly.

nVidia Titan X: Pascal GPGPU Performance in CUDA and OpenCL

By Adrian Silasi | 30th June 2018 - 10:44 pm |31st July 2018 Reviews

Comments Off

What is “Titan X (Pascal)”?

It is the current high-end “pro-sumer” card from nVidia using the current generation “Pascal” architecture – equivalent to the Series 10 cards. It is based on the 2nd-from-the-top 102 chipset (not the top-end 100) thus it does not feature full speed FP64/FP16 performance that is generally reserved for the “Quadro/Tesla” professional range of cards. It does however come with more memory to fit more datasets and is engineered for 24/7 performance.

Pricing has increased a bit from previous generation X/XP but that is a general trend today from all manufacturers.

See these other articles on Titan performance:

nVidia Titan V : Volta GPGPU performance in CUDA and OpenCL

Hardware Specifications

We are comparing the top-of-the-range Titan X with previous generation cards and competing architectures with a view to upgrading to a mid-range high performance design.

GPGPU Specifications	nVidia Titan X (P)	nVidia 980 GTX (M2)	AMD Vega 56	AMD Fury	Comments
Arch Chipset	Pascal GP102 (6.1)	Maxwell 2 GM204 (5.2)	Vega 10	Fiji	The X uses the current Pascal architecture that is also powering the current Series 10 consumer cards
Cores (CU) / Threads (SP)	28 / 3584	16 / 2048	56 / 3584	64 / 4096	We’ve got 28CU/SMX here down from 32 on GP100/Tesla but should still be sufficient to power through tasks.
FP32 / FP64 / Tensor Cores	3584 / 112 / no	2048 / 64 / no	3584 / 448 / no	4096 / 512 / no	Only 112 FP64 units – a lot less than competition from AMD, this is a card geared for FP32 workloads.
Speed (Min-Turbo)	1.531GHz (139-1910)	1.126GHz (135-1.215)	1.64GHz	1GHz	Higher clocked that previous generation and comparative with competition.
Power (TDP)	250W (125-300)	180W (120-225)	200W	150W	TDP has also increased to 250W but again that is inline with top-end cards that are pushing over 200W.
ROP / TMU	96 / 224	64 / 128	64 / 224	64 / 256	As it may also be used as top-end graphics card, it has a good amount of ROPs (50% more than competition) and similar numbers of TMUs.
Global Memory	12GB GDDR5X 10Gbps 384-bit	4GB GDDR5 7Gbps 256-bit	8GB HBM2 2Gbps 2048-bit	4GB HBM 1Gbps 4096-bit	Titan X comes with a huge 12GB of current GDDR5X memory while the competition has switched to HBM2 for top-end cards.
Memory Bandwidth (GB/s)	512	224	483	512	Due to high speed GDDR5X, the X has plenty of memory bandwidth even higher than HBM2 competition.
L2 Cache	3MB	2MB			L2 cache has increased by 50% over previous arch to keep all cores fed.
FP64/double ratio	1/32	1/32	1/8	1/8	The X is not really meant for FP64 workloads as it uses the same ratio 1:32 as normal consumer cards.
FP16/half ratio	1/64	n/a	1/1	1/1	With 1:64 ratio FP16 is not really usable on Titan X but can only really be used for compatibility testing.

Processing Performance

We are testing both CUDA native as well as OpenCL performance using the latest SDK / libraries / drivers from both nVidia and competition.

Results Interpretation: Higher values (GOPS, MB/s, etc.) mean better performance.

Environment: Windows 10 x64, latest nVidia drivers 398.36, CUDA 9.2, OpenCL 1.2. Turbo / Boost was enabled on all configurations.

Motion-Blur (7×7) Filter single/FP32 (MPix/s)4,830 / 3,6201,910 / 1,440

Again same algorithm but even more data shared the V is 2x faster than the X.

Processing Benchmarks		nVidia Titan X CUDA/OpenCL	nVidia GTX 980 CUDA/OpenCL	AMD Vega 56 OpenCL	AMD Fury OpenCL	Comments

	Mandel FP32/Single (Mpix/s)	17,870 [37%] / 16,000	7,000 / 6,100	13,000	8,720	Titan X makes a good start beating the Vega by almost 40%.
	Mandel FP16/Half (Mpix/s)	245 [-98%] / n/a	n/a	13,130	7,890	FP16 is so slow that it is unusable – just for testing.
	Mandel FP64/Double (Mpix/s)	661 [-47%] / 672	259 / 265	1,250	901	FP64 is also quite slow though a lot faster than on the GTX 980.
	Mandel FP128/Quad (Mpix/s)	25 [-67%] / 24	10.8 / 10.7	77.3	55	Emulated FP128 precision depends entirely on FP64 performance and thus is… slow.
With FP32 “normal” workloads Titan X is quite fast, ~40% faster than Vega and about 2.5x faster than an older GTX 980 thus quite an improvement. But FP16 workloads should not apply – better off with FP32 – and FP64 is also about 1/2 the performance of a Vega – also slower than even a Fiji. As long as all workloads are FP32 there should be no problems.

	Crypto AES-256 (GB/s)	40 [-38%] / 38	16 / 16	65	46	Titan X is a lot faster than previous gen but still ~40% slower than a Vega
	Crypto AES-128 (GB/s)	52 [-38%] / 51	23 / 21	84	60	Nothing changes here , the X still about 40% slower than a Vega.

	Crypto SHA2-256 (GB/s)	134 [+4%] / 142	58 / 59	129	82	In this integer workload, somehow Titan X manages to beat the Vega by 4%!
	Crypto SHA1 (GB/s)	107 [-34%] / 114	50 / 54	163	124	SHA1 is mysteriously slower thus the X is ~35% slower than a Vega.
	Crypto SHA2-512 (GB/s)	72 [+2.3x] / 42	32 / 24	31	13.8	With 64-bit integer workload, Titan X is a massive 2.3x times faster than a Vega.
Historically, nVidia cards have not been tuned for integer workloads, but Titan X still manages to beat a Vega – the “gold standard” for crypto-currency hashing – on both SHA256 and especially on 64-bit integer SHA2-512! Perhaps for the first time a nVidia card is competitive on integer workloads and even much faster on 64-bit integer workloads.

	Black-Scholes float/FP32 (MOPT/s)	11,480 [+28%] / 11,470	5,280 / 5,280	9,000	11,220	In this FP32 financial workload Titan X is almost 30% faster than a Vega.
	Black-Scholes double/FP64 (MOPT/s)	1,370 [-36%] / 1,300	547 / 511	1,850	1,290	Switching to FP64 code, the X remains competitive and is about 35% slower.
	Binomial float/FP32 (kOPT/s)	2,240 [-8%] / 2,240	1,200 / 1,140	2,440	1,760	Binomial uses thread shared data thus stresses the SMX’s memory system and here Vega surprisingly does better by 8%
	Binomial double/FP64 (kOPT/s)	129 [-20%] / 133	51 / 51	161	115	With FP64 code the X is only 20% slower than a Vega.
	Monte-Carlo float/FP32 (kOPT/s)	5,350 [+47%] / 5,150	2,140 / 2,000	3,630	2,470	Monte-Carlo also uses thread shared data but read-only thus reducing modify pressure – here Titan X is almost 50% faster!
	Monte-Carlo double/FP64 (kOPT/s)	294 [-34%] / 267	118 / 106	385	332	Switching to FP64 the X is again 34% slower than a Vega.
For financial FP32 workloads, the Titan X generally beats the Vega by a good amount or at least ties with it; with FP64 precision it is about 1/2 the speed which is to be expected. As long as you have FP32 workloads this should not be a problem.

	SGEMM (GFLOPS) float/FP32	6,280 [+19%] / 6,600	2,550 / 2,550	5,260	3,630	Using 32-bit precision Titan X beats the Vega by 20%.
	DGEMM (GFLOPS) double/FP64	335 [-40%] / 332	130 / 129	555	381	With FP64 precision, unsurprisingly the X is 40% slower.
	SFFT (GFLOPS) float/FP32	242 [-20%] / 227	148 / 136	306	348	FFT does better with HBM memory and here Titan X is 20% slower than a Vega.
	DFFT (GFLOPS) double/FP64	207 / 191	89 / 82	139	116	Surprisingly the X does very well here and manages to beat all cards by almost 50%!
	SNBODY (GFLOPS) float/FP32	5,600 [+20%] / 4,870	2,100 / 2,000	4,670	3,080	Titan X does well in this algorithm, beating the Vega by 20%.
	DNBODY (GFLOPS) double/FP64	275 [-20%] / 275	82 / 81	343	303	With FP64 precision, the X is again 20% slower.
The scientific scores are similar to the financial ones but the gain/loss is about 20% not 40% – in FP32 workloads Titan X is 20% faster while in FP64 it is about 20% slower than a Vega – a closer result than expected.

	Blur (3×3) Filter single/FP32 (MPix/s)	14,550 [-60%] / 10,880	7,310 / 5,530	36,000	28,000	In this 3×3 convolution algorithm, somehow Titan X is over 50% slower than a Vega and even a Fury.
	Sharpen (5×5) Filter single/FP32 (MPix/s)	3,840 [-11%] / 2,750	1,870 / 1,380	4,300	3,150	Same algorithm but more shared data reduces the gap to 10% but still a loss.
	Motion Blur (7×7) Filter single/FP32 (MPix/s)	3,920 [-10%] / 2,930	1,910 / 1,440	4,350	3,200	With even more data the gap remains at 10%.
	Edge Detection (2*5×5) Sobel Filter single/FP32 (MPix/s)	3,740 [-11%] / 2,760	1,860 / 1,370	4,210	3,130	Still convolution but with 2 filters – Titan X is 10% slower again.
	Noise Removal (5×5) Median Filter single/FP32 (MPix/s)	35.7 / 55 [+52%]	20.6 / 25.4	36.3	30.8	Different algorithm allows the X to finally beat the Vega by 50%.
	Oil Painting Quantise Filter single/FP32 (MPix/s)	15.6 [-60%] / 15.3	12.2 / 11.4	38.7	14.3	Without major processing, this filter does not like the X much it runs 1/2 slower than the Vega.
	Diffusion Randomise (XorShift) Filter single/FP32 (MPix/s)	16,480 [-57%] / 14,000	7,600 / 6,640	38,730	28,500	This algorithm is 64-bit integer heavy but again Titan X is 1/2 the speed of Vega.
	Marbling Perlin Noise 2D Filter single/FP32 (MPix/s)	290 / 6,350 [+13%]	210 / 3,080	5,600	4,410	One of the most complex and largest filters, Titan X finally beats the Vega by over 10%.
For image processing using FP32 precision Titan X surprisingly does not do as well as expected – either in CUDA or OpenCL – with the Vega beating it by a good margin on most filters – a pretty surprising result. Perhaps more optimisations are needed on nVidia hardware. We obviously did not test FP16 performance at all as that would have been far slower.

Memory Performance

We are testing both CUDA native as well as OpenCL performance using the latest SDK / libraries / drivers from nVidia and competition.

Results Interpretation: For bandwidth tests (MB/s, etc.) high values mean better performance, for latency tests (ns, etc.) low values mean better performance.

Environment: Windows 10 x64, latest nVidia drivers 398.36, CUDA 9.2, OpenCL 1.2. Turbo / Boost was enabled on all configurations.

Memory Benchmarks		nVidia Titan X CUDA/OpenCL	nVidia GTX 980 CUDA/OpenCL	AMD Vega 56 OpenCL	AMD Fury OpenCL	Comments

	Internal Memory Bandwidth (GB/s)	356 [+13%] / 354	145 / 144	316	387	Titan X brings more bandwidth than a Vega (+13%) but the old Fury takes the crown.
	Upload Bandwidth (GB/s)	11.4 / 9	12.1 / 12	12.1	11	All cards use PCIe3 x16 and thus no appreciable delta.
	Download Bandwidth (GB/s)	12.2 / 8.9	11.5 / 12.2	10	9.8	Again no significant difference but we were not expecting any.
Titan X uses current GDDR5X but with high data rate allowing it to bring more bandwidth that some HBM2 designs – a pretty impressive feat. Naturally high-end cards using HBM2 should have even higher bandwidth.

	Global (In-Page Random Access) Latency (ns)	201 / 230	230	273	343	Compared to previous generation, Titan X has better latency due to higher data rate.
	Global (Full Range Random Access) Latency (ns)	286 / 311	306	399	525	Similarly, even full random accesses are faster,
	Global (Sequential Access) Latency (ns)	89 / 121	97	129	216	Sequential access has similarly low latencies but nothing special.
	Constant Memory (In-Page Random Access) Latency (ns)	117 / 174	126	269	353	Constant memory latencies have also dropped.
	Shared Memory (In-Page Random Access) Latency (ns)	18 / 53	21	49	112	Even shared memory latencies have dropped likely due to higher core clocks.
	Texture (In-Page Random Access) Latency (ns)	195 / 196	208		121	Texture access latencies have come down as well.
	Texture (Full Range Random Access) Latency (ns)	282 / 278	308			And even full range latencies have decreased.
	Texture (Sequential Access) Latency (ns)	87 /123	102			With sequential access there is no appreciable delta in latencies.
We’re only comparing CUDA latencies here (as OpenCL is quite variable) – thus compared to the previous generation (GTX 980) all latencies are down, either due to higher memory data rate or core clock increases – but nothing spectacular. Still good progress and everything helps.

We see L1 cache effects until 16kB (same as previous arch) and between 2-4MB tallying with the 3MB cache. While fast perhaps they could be a bit bigger.

As with global memory we see the same L1D and L2 cache affects with similar latencies. All in all good performance but we could do with bigger caches.

Titan X’s memory performance is what you’d expect from higher clocked GDDR5X memory – it is competitive even with the latest HBM2 powered competition – both bandwidth and latency wise. There are no major surprises here and everything works nicely.

SiSoftware Official Ranker Scores

Final Thoughts / Conclusions

Titan X based on the current “Pascal” architecture performs very well in FP32 workloads – it is much faster than previous generation for a modest price increase and is competitive with the AMD’s Vega offers. But it is likely due to be replaced soon as next-generation “Volta” architecture is already out on the high-end (Titan V) and likely due to filter down the stack to both consumer (Series 11?) cards and “pro-sumer” cheaper Titan cards than the Titan V.

For FP64 workloads it is perhaps best to choose an older Quadro/Tesla card with more FP64 units as performance is naturally much lower. FP16 performance is also restricted and pretty much not usable – good for compatibility testing should you hope to upgrade to a full-speed FP16 card in the future. For both these workloads – the high-end Titan V is the card you probably want – but at a much higher price.

Still for the money, Titan X has its place and the most common FP32 workloads (financial, scientific, high precision image processing, etc.) that do not require FP64 nor FP16 optimisations perform very well and this card is all you need.