Simulating high latency in load test - visual-studio

I have a Visual Studio web performance test that I am planning on running with a variety of network mixes to simulate different network conditions. However when I report this I would like to know context of this (actual bandwidth, latency in ms, etc.). The best information I've found is this: http://msdn.microsoft.com/en-us/library/dd997557.aspx
Specifically what I would like to know is: What is the intra-continental connection's properties?
Is there a better reference on this?

As said by AdrianHHH, you can find *.network file for each of available network profile at %ProgramFiles%\Microsoft Visual Studio XXX\Common7\IDE\Templates\LoadTest\Networks e.g.IntracontinentalWAN.network. This file contains all settings for a network file like Latency, Packet Loss, Queue Management etc.
A good description of all properties are available here. There is not problem to edit an existing profile and to create a new one just for your specific
So, the intra-continental connection's properties are
<NetworkEmulationProfile name="Intra-continental WAN 1.5 Mbps" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Emulation>
<VirtualChannel name="ICWAN-Channel2">
<FilterList />
<VirtualLink instances="1" name="link1">
<LinkRule dir="upstream">
<Bandwidth>
<Speed unit="kbps">1500</Speed>
<QueueManagement>
<NormalQueue>
<Size>100</Size>
<QueueMode>packet</QueueMode>
<DropType>DropTail</DropType>
</NormalQueue>
</QueueManagement>
</Bandwidth>
<Latency>
<Fixed>
<Time unit="msec">50</Time>
</Fixed>
</Latency>
</LinkRule>
<LinkRule dir="downstream">
<Bandwidth>
<Speed unit="kbps">1500</Speed>
<QueueManagement>
<NormalQueue>
<Size>100</Size>
<QueueMode>packet</QueueMode>
<DropType>DropTail</DropType>
</NormalQueue>
</QueueManagement>
</Bandwidth>
<Latency>
<Fixed>
<Time unit="msec">50</Time>
</Fixed>
</Latency>
</LinkRule>
</VirtualLink>
</VirtualChannel>
</Emulation>
</NetworkEmulationProfile>

Related

What can be done to lower UE4Editor startup time?

Status: the problem lowered, but compared to other users reports it persists.
I have moved to UE4.27.0 and the startup time lowered from 11 (v4.26.2) to 6 minutes! (the RAM usage lowered too!) But doesnt compare to the speed other ppl report "almost instantly"...
It is not compiling anything, not even shaders, it is like the 6th time I run it for one project.
Should I try to disable plugins? but Im new with UE and dont want to difficult my usage. Tho, for ex., I have nothing VR related to test so it could really be initially disabled.
HD READ SPEED? NO
I have tested moving UE4Editor whole engine path (100GB) to a 3xSSD(Stripes), but the UE4Editor startup time remained the same. My HD were it is too, is fast but not so fast as the 3xSSD.
CPU USAGE? MAY BE if it could use 4 cores could solve it?
UE4Editor startup uses A SINGLE CORE ONLY, i can confirm with htop and system monitor, it is possible to see only a single core being used 100% and it changes between the 4 cores, so only one is used at 100% per time.
I tested this command line parameter -USEALLAVAILABLECORES after the project URL for UE4Editor, but nothing changed. I read that option is ignored in some machines, so may be if I patch it's usage it could work on mine?
GPU? no?
a report about an integrated graphics card (weak one) says it doesnt interfere with the startup time.
LOG for UE4Editor v4.27.0 with the new biggest intervals ("..." means ommited log lines to make it easier to read; "!(interval in seconds)" is just to easy reading it (no ommitted lines here)):
[2021.09.15-23.38.20:677][ 0]LogHAL: Linux SourceCodeAccessSettings: NullSourceCodeAccessor
!22s
[2021.09.15-23.38.42:780][ 0]LogTcpMessaging: Initializing TcpMessaging bridge
[2021.09.15-23.38.42:782][ 0]LogUdpMessaging: Initializing bridge on interface 0.0.0.0:0 to multicast group 230.0.0.1:6666.
!16s
[2021.09.15-23.38.58:158][ 0]LogPython: Using Python 3.7.7
...
[2021.09.15-23.39.01:817][ 0]LogImageWrapper: Warning: PNG Warning: Duplicate iCCP chunk
!75s
[2021.09.15-23.40.16:951][ 0]SourceControl: Source control is disabled
...
[2021.09.15-23.40.26:867][ 0]LogAndroidPermission: UAndroidPermissionCallbackProxy::GetInstance
!16s
[2021.09.15-23.40.42:325][ 0]LogAudioCaptureCore: Display: No Audio Capture implementations found. Audio input will be silent.
...
[2021.09.15-23.41.08:207][ 0]LogInit: Transaction tracking system initialized
!9s
[2021.09.15-23.41.17:513][ 0]BlueprintLog: New page: Editor Load
!23s
[2021.09.15-23.41.40:396][ 0]LocalizationService: Localization service is disabled
...
[2021.09.15-23.41.45:457][ 0]MemoryProfiler: OnSessionChanged
!13s
[2021.09.15-23.41.58:497][ 0]LogCook: Display: CookSettings for Memory: MemoryMaxUsedVirtual 0MiB, MemoryMaxUsedPhysical 16384MiB, MemoryMinFreeVirtual 0MiB, MemoryMinFreePhysical 1024MiB
SPECS:
I'm using ubuntu 20.04.
My CPU is 4 cores 3.6GHz.
GeForce GT 710 1GB.
Related question but for older UE4: https://answers.unrealengine.com/questions/987852/view.html
Unreal Engine needs a high-end pc with a lot of RAM, fast SSD's, a good CPU and a medium graphic card. First of all there are always some shaders that needs to be compiled from the engine, and a lot of assets to be loaded in the startup time. As I can see you're on Linux you are probably using a self-compiled Unreal Engine version.... not the best thing to do for a newbie, because this may cause several problems on load time, startup, compiling and a lot of other stuff. If it's the first times you're using Unreal, try using it on Windows, it's all easier.

How to identify PC (motherboard) in win32 api? [duplicate]

How to uniquely identify computer (mainboard) using C#(.Net/Mono, local application)?
Edition. We can identify mainboard in .Net using something like this (see Get Unique System Identifiers in C#):
using System.Management;
...
ManagementObjectSearcher searcher = new ManagementObjectSearcher("select * from Win32_MotherboardDevice");
...
But unfortunately Mono does not support System.Management. How to do it under Mono for Linux? - I don't know :(
Write a function that takes a few unique hardware parameters as input and generates a hash out of them.
For example, Windows activation looks at the following hardware characteristics:
Display Adapter
SCSI Adapter
IDE Adapter (effectively the motherboard)
Network Adapter (NIC) and its MAC Address
RAM Amount Range (i.e., 0-64mb, 64-128mb, etc.)
Processor Type
Processor Serial Number
Hard Drive Device
Hard Drive Volume Serial Number (VSN)
CD-ROM / CD-RW / DVD-ROM
You can pick up a few of them to generate your unique computer identifier.
Please see: Get Unique System Identifiers in C#
You realistically have MotherboardID, CPUID, Disk Serial and MAC address, from experience none of them are 100%.
Our stats show
Disk serial Is missing 0.1 %
MAC Is missing 1.3 %
Motherboard ID Is missing 30 %
CPUID Is missing 99 %
0.04% of machines tested yielded no information, we couldn't even read the computer name. It maybe that these were some kind of virtual PC, HyperV or VMWare instance, or maybe just very locked down? In any case your design has to be able to cope with these cases.
Disk serial is the most reliable, but easy to change, mac can be changed and depending on the filtering applied when reading it can change if device drivers are added (hyperv, wireshark etc).
Motherboard and CPUID sometimes return values that are invalid "NONE", "AAAA..", "XXXX..." etc.
You should also note that these functions can be very slow to call (they may take a few seconds even on a fast PC), so it may be worth kicking them off on a background thread as early as possible, you ideally don't want to be blocking on them.
Try this:
http://carso-owen.blogspot.com/2007/02/how-to-get-my-motherboard-serial-number.html
Personally though, I'd go with hard drive serial number. If a mainboard dies and is replaced, that PC isn't valid any more. If the HDD drive is replaced, it doesn't matter too much because the software was on it.
Of course, on the other hand, if the HDD is just moved elsewhere, the information goes with it, so you might want to look at a combination of serial numbers, depending what you want it for.
How about the MAC address of the network card?

The difference between exclude_hv and exclude_host in perf

In kernel 3.11.0, in the struct perf_event_attr, there are three members named exclude_hv/exclude_host/exclude_guest.
I know the exclude_host field is to exclude events generated by the host when running kvm. But what is the meaning of exclude_hv? Is it used in the Xen?
What is the mechanism in hardware that supports the function of exclude_host? As far as I know, in the performance monitoring select registers, there are no such bits that control the event counter to exclude events generated by the host.
This is a bit old but for those looking at the answer, as me:
exclude_hv: do not count events that occur in the hypervisor.
The distinction between events occurred in user space, kernel, hypervisor, host, etc is done in software. The kernel and/or hypervisor will retire and replace the event count and configuration on each change of context.
Here is an excellent description of perf_events, which is the kernel module that handles the performance counters.

Can ETW (event tracing for windows) be used to gather also memory statistics?

Is it possible using ETW to also get memory statistics of all the processes and the system ?
With memory statistics I mean : e.g. Commited bytes, private bytes,paged pool,working set,...
I cannot find anything about using xperf to get and see memory statistics. It is always about CPU , disk , network.
One could probably use performance counters to get that kind of information, but how can one overlay the statistics graphically in one chart (how to correlate/sync the timestamps) ?
Your best bet on Windows 8.1 and higher is the Microsoft-Windows-Kernel-Memory provider, which records per-process memory information every 0.5 s. See https://github.com/google/UIforETW/issues/80 for details. UIforETW enables this by default when it is available.
You could also try the MEMINFO provider. It gives a system-wide overview of memory pressure. It shows the Active List (currently in use memory), the Standby List ('useful' pages not currently in use, such as the disk cache), and the Zero and Free lists (genuinely free memory). This at least lets you tell whether a system is running out of memory.
You could also try MEMINFO_WS and CONTMEMGEN but these are undocumented so I really don't know what they do. They show up in xperf -providers k but when I record with them I can't see any new graphs appearing. Apparently Microsoft ships these providers but no way to view them. Sigh...
If you want more memory details on Windows 7 -- such as per-process working sets -- your best bet is to have a process running which periodically queries this data and emits it in custom ETW events. This is available in a prepackaged form in UIforETW which can query the working set of a specified set of processes once a second. See the announcement post for how to get UIforETW:
https://randomascii.wordpress.com/2015/04/14/uiforetw-windows-performance-made-easier/
UIforETW's Windows 7 working set data shows up in Generic Events under Task Name == WorkingSet. On Windows 8.1 the OS working set data (more detailed, more efficiently recorded) shows up under Memory-> Virtual Memory Snapshots.
You can trace memory usage with ReferenceSet kernel group. It includes the following traceflags:
PROC_THREAD+LOADER+HARD_FAULTS+MEMORY+FOOTPRINT+VIRT_ALLOC+MEMINFO+VAMAP+SESSION+REFSET+MEMINFO_WS
MEMORY = Memory tracing
FOOTPRINT+REFSET = Support footprint analysis
MEMINFO = Memory List Info (active, standby and oters you see from ResMon)
VIRT_ALLOC = Virtual allocation reserve and release
VAMAP = mapped files information
MEMINFO_WS = Working set Info
As you can see xperf can capture a lot of memory data when you sue the right flags.

What does the term POLICED in Linux network scheduler (tc qdisc) named after?

POLICED means that the speed of ingress packet will be limited and the exceeding packets will be dropped.
I saw a comment "skb is shot by police" in include/linux/netdevice.h.
It's of course alike, but is there any story behind this name?
Yes -
http://lxr.linux.no/#linux+v3.2.9/net/sched/Kconfig
menuconfig NET_SCHED
bool "QoS and/or fair queueing"
select NET_SCH_FIFO
---help---
When the kernel has several packets to send out over a network
device, it has to decide which ones to send first, which ones to
delay, and which ones to drop. This is the job of the queueing
disciplines, several different algorithms for how to do this
"fairly" have been proposed.
...
config NET_ACT_POLICE
tristate "Traffic Policing"
depends on NET_CLS_ACT
---help---
Say Y here if you want to do traffic policing, i.e. strict
bandwidth limiting. This action replaces the existing policing
module.
To compile this code as a module, choose M here: the
module will be called act_police.

Resources