{"id":5289,"date":"2024-01-24T13:29:23","date_gmt":"2024-01-24T20:29:23","guid":{"rendered":"https:\/\/blog.caida.org\/best_available_data\/?p=5289"},"modified":"2024-01-24T13:29:23","modified_gmt":"2024-01-24T20:29:23","slug":"developing-software-locally-for-ark","status":"publish","type":"post","link":"https:\/\/blog.caida.org\/best_available_data\/2024\/01\/24\/developing-software-locally-for-ark\/","title":{"rendered":"Developing active Internet measurement software locally to run on Ark"},"content":{"rendered":"

In the first part of our blog series<\/a>, we introduced our brand-new python module for scamper<\/a>, the packet-prober underpinning much of Ark’s ongoing measurements<\/a>. One aspect that we highlighted was the ability for potential users of Ark to develop their code locally, before running it on the Ark platform. When I develop measurement applications, I use a couple of local Raspberry Pis and my own workstation to get the code correct, and then copy the code to the CAIDA system to run the experiment using available Ark vantage points. The goal of this blog article is to describe different ways to locally develop your measurement experiment code.<\/p>\n

Example #1: Starting small with one scamper process.<\/strong><\/p>\n

The easiest way to begin is with one scamper process running on the same system where you develop your python code. With scamper installed (we recommend that you use a package listed on the scamper website<\/a>), start a scamper process, and make it available for measurement commands on a Unix domain socket. For example, you might run scamper as follows:<\/p>\n

$ scamper -U \/tmp\/scamper -p 100<\/code><\/p>\n

This will create a Unix domain socket to drive scamper at \/tmp\/scamper, and tell scamper that it can probe at up to 100 packets\/second. You can adjust these parameters to what is appropriate locally.<\/p>\n

You can then develop and debug your measurement code in Python. To use this scamper process, your Python code might begin as follows:<\/p>\n

01 from<\/span> scamper import<\/span> ScamperCtrl
\n02
\n03 # use the scamper process available at \/tmp\/scamper
\n04 ctrl<\/span> = ScamperCtrl(unix=\"\/tmp\/scamper\"<\/span>)
\n05
\n06 # do a simple ping to 8.8.8.8 and print the outcome
\n07 o<\/span> = ctrl.do_ping(\"8.8.8.8\"<\/span>)
\n08 if<\/span> o.min_rtt is not<\/span> None<\/span>:
\n09\u00a0\u00a0 print<\/span>(f\"{o.min_rtt.total_seconds()*1000):.1f} ms\"<\/span>)
\n10 else<\/span>:
\n11\u00a0\u00a0 print<\/span>(\"no reply\"<\/span>)<\/code><\/p>\n

Example #2: Coordinating measurements among VPs.<\/strong><\/p>\n

Once you are comfortable using the python module with a single local scamper instance, you might want to test your code with multiple scamper instances, each representing a distinct vantage point. The scamper software includes the sc_remoted<\/a> interface to support that. sc_remoted has features for authenticating endpoints with TLS, but you might choose to initially operate endpoints without the complexity of TLS.<\/p>\n

sc_remoted listens on a port for inbound scamper connections, and makes Unix domain sockets — one for each VP — available in a nominated directory. The best idea is to create an empty directory for these sockets. You might run sc_remoted as follows:<\/p>\n

$ mkdir -p \/path\/to\/remote-sockets
\n$ sc_remoted -U \/path\/to\/remote-sockets -P 50265
\n<\/code><\/p>\n

The first command creates the directory, because sc_remoted will not create that directory for you. The second command starts sc_remoted listening on port 50265 for incoming scamper connections, and will place Unix domain sockets in \/path\/to\/remote-sockets as they arrive.\u00a0 Note, we use \/path\/to as a placeholder<\/a> to the actual path in your local file system that is appropriate for your environment; you might put these sockets in a directory somewhere in your home directory, for example.<\/p>\n

Then, on the systems that you want to act as vantage points, the following command:<\/p>\n

$ scamper -p 100 -R 192.0.2.28:50265 -M foo.bar<\/code><\/p>\n

will (1) start a scamper process, (2) tell it that it can probe at up to 100 packets-per-second, (3) connect it to the specified IP address and port to receive measurement commands from, and (4) tell it to identify itself as “foo.bar” to the remote controller. If you go into \/path\/to\/remote-sockets, you might see the following:<\/p>\n

$ cd \/path\/to\/remote-sockets
\n$ ls -l
\ntotal 0
\nsrwx------ 1 mjl mjl 0 Jan 22 16:57 foo.bar-192.0.2.120:12369<\/code><\/p>\n

This socket represents the scamper process you just started. The filename begins with foo.bar, the parameter that you gave to scamper to identify itself. After the dash is the IP address and port number that the remote controller observed the remote system coming from. You can connect as many additional scamper instances as you like, and you will see them listed in the directory individually. You should name each differently with something meaningful to you (foo.bar, bar.baz, etc) so that you can identify them on the system on which you’re writing your python code.<\/p>\n

The python code we wrote in Example #1 above might be modified as follows:<\/p>\n

01 import<\/span> sys
\n02 from<\/span> scamper import<\/span> ScamperCtrl
\n03
\n04 if<\/span> len(sys.argv) != 2:
\n05\u00a0\u00a0 print<\/span>(\"specify path to unix domain socket\"<\/span>)
\n06\u00a0\u00a0 sys.exit<\/span>(-1)
\n07
\n08 # use the remote scamper process available at the specified location
\n09 ctrl<\/span> = ScamperCtrl(remote=sys.argv[1])
\n10
\n11 # do a simple ping to 8.8.8.8 and print the outcome
\n12 o<\/span> = ctrl.do_ping(\"8.8.8.8\"<\/span>)
\n13 if<\/span> o.min_rtt is not<\/span> None<\/span>:
\n14\u00a0\u00a0 print<\/span>(f\"{o.min_rtt.total_seconds()*1000):.1f} ms\"<\/span>)
\n15 else<\/span>:
\n16\u00a0\u00a0 print<\/span>(\"no reply\"<\/span>)<\/code><\/p>\n

And run as:<\/p>\n

$ python ping.py \/path\/to\/remote-sockets\/foo.bar-192.0.2.120\\:12369<\/code><\/p>\n

If you have multiple remote-sockets in the directory, you can add them individually<\/a>, or use all sockets in the directory<\/a>. For example:<\/p>\n

01 import<\/span> sys
\n02 from<\/span> datetime import<\/span> timedelta
\n03 from<\/span> scamper import<\/span> ScamperCtrl
\n04
\n05 if<\/span> len(sys.argv) != 3:
\n06\u00a0\u00a0 print<\/span>(\"usage: single-radius.py $dir $ip\"<\/span>)
\n07\u00a0\u00a0 sys.exit<\/span>(-1)
\n08
\n09 ctrl<\/span> = ScamperCtrl(remote_dir=sys.argv[1])
\n10 for<\/span> i in<\/span> ctrl.instances():
\n11\u00a0\u00a0 ctrl.do_ping(sys.argv[2], inst=i)
\n12
\n13 min_rtt<\/span> = None<\/span>
\n14 min_vp<\/span> = None<\/span>
\n15 for<\/span> o in<\/span> ctrl.responses(timeout=timedelta(seconds=10)):
\n16\u00a0\u00a0 if<\/span> o.min_rtt is not<\/span> None<\/span> and<\/span> (min_rtt is<\/span> None<\/span> or<\/span> min_rtt > o.min_rtt):
\n17\u00a0\u00a0\u00a0\u00a0 min_rtt<\/span> = o.min_rtt
\n18\u00a0\u00a0\u00a0\u00a0 min_vp<\/span> = o.inst
\n19
\n20 if<\/span> min_rtt is not<\/span> None<\/span>:
\n21\u00a0\u00a0 print<\/span>(f\"{min_vp.name} {(min_rtt.total_seconds()*1000):.1f} ms\"<\/span>)
\n22 else<\/span>:
\n23\u00a0\u00a0 print<\/span>(f\"no responses for {sys.argv[2]}\"<\/span>)<\/code><\/p>\n

and run this command:<\/p>\n

$ python single-radius.py \/path\/to\/remote-sockets 8.8.8.8<\/code><\/p>\n

We encourage you to reach out via email if you have questions about using the module. In the first instance, you can email ark-info at caida.org.<\/p>\n","protected":false},"excerpt":{"rendered":"

In the first part of our blog series, we introduced our brand-new python module for scamper, the packet-prober underpinning much of Ark’s ongoing measurements. One aspect that we highlighted was the ability for potential users of Ark to develop their code locally, before running it on the Ark platform. When I develop measurement applications, I […]<\/p>\n","protected":false},"author":14,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"coauthors":[33],"_links":{"self":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5289"}],"collection":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/comments?post=5289"}],"version-history":[{"count":11,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5289\/revisions"}],"predecessor-version":[{"id":5301,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/posts\/5289\/revisions\/5301"}],"wp:attachment":[{"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/media?parent=5289"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/categories?post=5289"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/tags?post=5289"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.caida.org\/best_available_data\/wp-json\/wp\/v2\/coauthors?post=5289"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}