tag:blogger.com,1999:blog-91664537879439311532024-03-13T14:28:39.376-07:00Technical writeupAnonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-9166453787943931153.post-53352401041890928772016-08-08T22:56:00.001-07:002016-08-08T22:57:39.644-07:00How to install VMware tools on linux based vm<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div style="font-family: -webkit-standard; font-size: 14px;">
vSphere needs VMware tools to detect the IP of the vm. Following are the steps to install the VMware tools:
</div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b># To see the installed version </b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<table style="-evernote-table: true; border-collapse: collapse; font-family: -webkit-standard; font-size: 14px; margin-left: 0px; table-layout: fixed; width: 100%;">
<tbody>
<tr><td style="border-color: rgb(219,219,219); border-style: solid; border-width: 1px; margin: 0px; padding: 10px; width: 100%;"><div>
<span style="font-family: "courier new";">vmware-toolbox-cmd -v</span></div>
</td>
</tr>
</tbody></table>
<div style="font-family: -webkit-standard; font-size: 14px;">
</div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b># To see the currently installed packages</b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<table style="-evernote-table: true; border-collapse: collapse; font-family: -webkit-standard; font-size: 14px; margin-left: 0px; table-layout: fixed; width: 100%;">
<tbody>
<tr><td style="border-color: rgb(219,219,219); border-style: solid; border-width: 1px; margin: 0px; padding: 10px; width: 100%;"><div>
<span style="font-family: "courier new";">sudo apt --installed list | grep tools</span></div>
</td>
</tr>
</tbody></table>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b># Steps to remove open-vm-tools</b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<table style="-evernote-table: true; border-collapse: collapse; font-family: -webkit-standard; font-size: 14px; margin-left: 0px; table-layout: fixed; width: 100%;">
<tbody>
<tr><td style="border-color: rgb(219,219,219); border-style: solid; border-width: 1px; margin: 0px; padding: 10px; width: 100%;"><div>
<div>
<span style="font-family: "courier new";">sudo apt-get remove --purge open-vm-tools</span></div>
<div>
<span style="font-family: "courier new";">sudo apt-get -f install</span></div>
</div>
<div>
<span style="font-family: "courier new";">sudo apt-get autoremove</span></div>
</td>
</tr>
</tbody></table>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b># Steps to add vmware vm-tools </b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<table style="-evernote-table: true; border-collapse: collapse; font-family: -webkit-standard; font-size: 14px; margin-left: 0px; table-layout: fixed; width: 100%;">
<tbody>
<tr><td style="border-color: rgb(219,219,219); border-style: solid; border-width: 1px; margin: 0px; padding: 10px; width: 100%;"><div>
<div>
<span style="font-family: "courier new";">sudo add-apt-repository "deb http://packages.vmware.com/tools/esx/6.0u1/ubuntu precise main"</span></div>
<div>
<span style="font-family: "courier new";">sudo apt-get update</span></div>
<div>
<span style="font-family: "courier new";">sudo apt-get install vmware-tools-core</span></div>
<div>
<span style="font-family: "courier new";">sudo apt-get install vmware-tools-services</span></div>
</div>
<div>
<span style="font-family: "courier new";">sudo apt-get install vmware-tools-esx-nox</span></div>
</td>
</tr>
</tbody></table>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b># See the latest installed packages </b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<table style="-evernote-table: true; border-collapse: collapse; font-family: -webkit-standard; font-size: 14px; margin-left: 0px; table-layout: fixed; width: 100%;">
<tbody>
<tr><td style="border-color: rgb(219,219,219); border-style: solid; border-width: 1px; margin: 0px; padding: 10px; width: 100%;"><div>
<span style="font-family: "courier new";">sudo apt --installed list | grep tools</span></div>
</td>
</tr>
</tbody></table>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
</div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b># Verify latest version is installed</b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<table style="-evernote-table: true; border-collapse: collapse; font-family: -webkit-standard; font-size: 14px; margin-left: 0px; table-layout: fixed; width: 100%;">
<tbody>
<tr><td style="border-color: rgb(219,219,219); border-style: solid; border-width: 1px; margin: 0px; padding: 10px; width: 100%;"><div>
<span style="font-family: "courier new";">vmware-toolbox-cmd -v</span></div>
</td>
</tr>
</tbody></table>
<div style="font-family: -webkit-standard; font-size: 14px;">
<b><br /></b></div>
<div style="font-family: -webkit-standard; font-size: 14px;">
<br /></div>
</div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-39994904528987005842016-08-08T22:43:00.003-07:002016-08-08T22:47:03.969-07:00Bash script to monitor uptime, cpu, memory, swap memory, load average of linux system<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># Print system resources <br />echo "HOSTNAME: "`hostname`<br />echo "SYSTEM: "`uname -a`<br />echo "IP ADDRESS: "; ifconfig eth0<br />echo "ROUTE TABLE:"; route -n<br />echo ""</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># Start monitoring the system resources<br />while true<br />do <br /> echo ------ `date` ------<br /> top -b n1 | head -n 6<br /> sleep 60<br />done</span></div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-32019994606297911872016-04-29T12:48:00.003-07:002016-04-29T12:48:55.211-07:00Python code to read and write array to file in json format<div dir="ltr" style="text-align: left;" trbidi="on">
First install and import 'json' module<br />
<br />
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 12.0pt;"><span style="color: #cc7832; font-weight: bold;">import </span>json</pre>
<br />
<h4 style="text-align: left;">
Reading JSON object from file:</h4>
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 12.0pt;"><span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">read_json_from_file</span>(json_file):
<span style="color: #629755; font-style: italic;">"""</span>
<span style="color: #629755; font-style: italic;"> Read array of objects from a file in json format.</span>
<span style="color: #629755; font-style: italic;"> </span><span style="color: #629755; font-style: italic; font-weight: bold;">:param</span><span style="color: #629755; font-style: italic;"> json_file: input json file name</span>
<span style="color: #629755; font-style: italic;"> </span><span style="color: #629755; font-style: italic; font-weight: bold;">:return</span><span style="color: #629755; font-style: italic;">: array of objects from the json</span>
<span style="color: #629755; font-style: italic;"> """</span>
<span style="color: #629755; font-style: italic;"> </span><span style="color: #cc7832; font-weight: bold;">with </span><span style="color: #8888c6;">open</span>(json_file<span style="color: #cc7832;">, </span><span style="color: #a5c261;">'r'</span>) <span style="color: #cc7832; font-weight: bold;">as </span>f:
exp_string = json.load(f)
json_array = json.loads(exp_string)
<span style="color: #cc7832; font-weight: bold;">return </span>json_array</pre>
<br />
<br />
<h4 style="text-align: left;">
Writing JSON to the file:</h4>
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 12.0pt;"><span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">write_json_to_file</span>(array<span style="color: #cc7832;">, </span>out_file_name):
<span style="color: #629755; font-style: italic;">"""</span>
<span style="color: #629755; font-style: italic;"> Dump an array to file in json format.</span>
<span style="color: #629755; font-style: italic;"> </span><span style="color: #629755; font-style: italic; font-weight: bold;">:param</span><span style="color: #629755; font-style: italic;"> array: Array to write to the file</span>
<span style="color: #629755; font-style: italic;"> </span><span style="color: #629755; font-style: italic; font-weight: bold;">:param</span><span style="color: #629755; font-style: italic;"> out_file_name: output file name</span>
<span style="color: #629755; font-style: italic;"> """</span>
<span style="color: #629755; font-style: italic;"> </span><span style="color: #cc7832; font-weight: bold;">with </span><span style="color: #8888c6;">open</span>(out_file_name<span style="color: #cc7832;">, </span><span style="color: #a5c261;">'w'</span>) <span style="color: #cc7832; font-weight: bold;">as </span>out_f:
s = json.dumps([o.<span style="color: #b200b2;">__dict__ </span><span style="color: #cc7832; font-weight: bold;">for </span>o <span style="color: #cc7832; font-weight: bold;">in </span>array]<span style="color: #cc7832;">, </span><span style="color: #aa4926;">sort_keys</span>=<span style="color: #8888c6;">True</span>)
json.dump(s<span style="color: #cc7832;">, </span>out_f)</pre>
<br /></div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-75596672208248710792016-04-29T12:26:00.000-07:002016-04-29T12:27:16.523-07:00Get recursive file listing in python<div dir="ltr" style="text-align: left;" trbidi="on">
Below is the python code:<br />
<br />
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 12.0pt;"><span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">get_recursive_filelist</span>(rootdir):
outfiles = []
<span style="color: #cc7832; font-weight: bold;">if </span>os.path.isfile(rootdir):
outfiles.append(rootdir)
<span style="color: #cc7832; font-weight: bold;">for </span>(dir<span style="color: #cc7832;">, </span>_<span style="color: #cc7832;">, </span>files) <span style="color: #cc7832; font-weight: bold;">in </span>os.walk(rootdir):
<span style="color: #cc7832; font-weight: bold;">for </span>f <span style="color: #cc7832; font-weight: bold;">in </span>files:
path = os.path.join(dir<span style="color: #cc7832;">, </span>f)
<span style="color: #cc7832; font-weight: bold;">if </span>os.path.exists(path):
outfiles.append(path)
<span style="color: #cc7832; font-weight: bold;">return </span>outfiles</pre>
</div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-3507053001545453652016-04-29T09:56:00.002-07:002016-04-29T09:56:37.958-07:00Easy way to check internet speed using command line speedtest tool<div dir="ltr" style="text-align: left;" trbidi="on">
Here is the light weight command line speedtest tool to check the internet speed.<br />
<span style="background-color: #cccccc;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>
<pre><span style="font-size: small;"><span style="background-color: #cccccc;"><span style="font-family: "Courier New",Courier,monospace;"><code>$ wget -O speedtest-cli https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py
$ chmod +x speedtest-cli
$ ./speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from Comcast Cable (x.x.x.x)...
Selecting best server based on ping...
Hosted by FiberCloud, Inc (Seattle, WA) [12.03 km]: 44.028 ms
Testing download speed........................................
Download: 32.29 Mbit/s
Testing upload speed..................................................
Upload: 5.18 Mbit/s</code></span></span></span></pre>
</div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com1tag:blogger.com,1999:blog-9166453787943931153.post-4090895906639614852016-04-29T09:48:00.000-07:002016-04-29T09:59:07.896-07:00Easy way to parse command line arguments in python using argparse<div dir="ltr" style="text-align: left;" trbidi="on">
Here is the easiest way to parse command line arguments in python.<br />
<br />
<br />
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 12.0pt;"><span style="color: #cc7832; font-weight: bold;">import </span>argparse
args = argparse.ArgumentParser(<span style="color: #aa4926;">description</span>=<span style="color: #a5c261;">'Description of the script.'</span>)
args.add_argument(<span style="color: #a5c261;">'-i'</span><span style="color: #cc7832;">, </span><span style="color: #a5c261;">'--input'</span><span style="color: #cc7832;">, </span><span style="color: #aa4926;">help</span>=<span style="color: #a5c261;">'Input file name'</span><span style="color: #cc7832;">, </span><span style="color: #aa4926;">required</span>=<span style="color: #8888c6;">True</span>)
args.add_argument(<span style="color: #a5c261;">'-o'</span><span style="color: #cc7832;">, </span><span style="color: #a5c261;">'--output'</span><span style="color: #cc7832;">, </span><span style="color: #aa4926;">help</span>=<span style="color: #a5c261;">'Output file'</span><span style="color: #cc7832;">, </span><span style="color: #aa4926;">required</span>=<span style="color: #8888c6;">True</span>)
args = args.parse_args()
input = args.input
output = args.output
<span style="color: #cc7832; font-weight: bold;">print </span><span style="color: #a5c261;">"Input:" </span>+ input
<span style="color: #cc7832; font-weight: bold;">print </span><span style="color: #a5c261;">"Ouput:" </span>+ output</pre>
<br />
<br />
<br />
<b>Output:</b><br />
<pre><span style="font-family: "courier new" , "courier" , monospace;">Rahuls-MacBook-Pro:exception rahuldiyewar$ python test.py -h
usage: test.py [-h] -i INPUT -o OUTPUT
Description of the script.
optional arguments:
-h, --help show this help message and exit
-i INPUT, --input INPUT
Input file name
-o OUTPUT, --output OUTPUT
Output file</span>
</pre>
<br />
<span style="font-family: "Courier New",Courier,monospace;">Rahuls-MacBook-Pro:exception rahuldiyewar$ python test.py -i myinput -o myoutput </span><br />
<span style="font-family: "Courier New",Courier,monospace;">Input:myinput </span><br />
<span style="font-family: "Courier New",Courier,monospace;">Ouput:myoutput
</span></div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-73028638846845594992016-04-29T09:26:00.000-07:002016-04-29T09:59:07.898-07:00Python code to search and create bug in JIRA using rest api<div dir="ltr" style="text-align: left;" trbidi="on">
Here is the Python code to search/open issues in JIRA:<br />
<ol style="text-align: left;">
<li>Search for open issues</li>
<li>Create new issue </li>
</ol>
<br />
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 12.0pt;"><span style="color: #cc7832; font-weight: bold;">from </span>jira <span style="color: #cc7832; font-weight: bold;">import </span>JIRA
<span style="color: #cc7832; font-weight: bold;">class </span><span style="font-weight: bold;">JIRARestAPI</span>:
url = <span style="color: #a5c261;">'https://<company-name>.atlassian.net'</span><span style="color: #a5c261;"> </span>
<span style="color: #a5c261;"> </span>username = <span style="color: #a5c261;">'<username>'</span><span style="color: #a5c261;"> </span>
<span style="color: #a5c261;"> </span>password = <span style="color: #a5c261;">'<password>'</span><span style="color: #a5c261;"> </span>
<span style="color: #a5c261;"> </span>jira = JIRA(url<span style="color: #cc7832;">, </span><span style="color: #aa4926;">basic_auth</span>=(username<span style="color: #cc7832;">, </span>password))
<span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">search_issues</span>(<span style="color: #94558d;">self</span><span style="color: #cc7832;">, </span>search_string):
<span style="color: #cc7832; font-weight: bold;">return </span>JIRARestAPI.jira.search_issues(search_string)
<span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">print_bugs</span>(<span style="color: #94558d;">self</span><span style="color: #cc7832;">, </span>bugs):
<span style="color: #cc7832; font-weight: bold;">for </span>bug <span style="color: #cc7832; font-weight: bold;">in </span>bugs:
<span style="color: #94558d;">self</span>.print_bug(bug)
<span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">print_bug</span>(<span style="color: #94558d;">self</span><span style="color: #cc7832;">, </span>bug):
<span style="color: #cc7832; font-weight: bold;">print </span>(<span style="color: #a5c261;">"%s, %s, </span><span style="color: #cc7832;">\"</span><span style="color: #a5c261;">%s</span><span style="color: #cc7832;">\"</span><span style="color: #a5c261;">" </span>% (bug.key<span style="color: #cc7832;">, </span>bug.fields.status.name<span style="color: #cc7832;">, </span>bug.fields.summary))
<span style="color: #cc7832; font-weight: bold;">def </span><span style="font-weight: bold;">create_cyg_bug</span>(<span style="color: #94558d;">self</span><span style="color: #cc7832;">, </span>summary<span style="color: #cc7832;">, </span>description<span style="color: #cc7832;">, </span>label):
issue_dict = {
<span style="color: #a5c261;">'project'</span>: {<span style="color: #a5c261;">'id'</span>: <span style="color: #6897bb;">10000</span>}<span style="color: #cc7832;">,</span>
<span style="color: #cc7832;"> </span><span style="color: #a5c261;">'summary'</span>: summary<span style="color: #cc7832;">,</span>
<span style="color: #cc7832;"> </span><span style="color: #a5c261;">'description'</span>: description<span style="color: #cc7832;">,</span>
<span style="color: #cc7832;"> </span><span style="color: #a5c261;">'issuetype'</span>: {<span style="color: #a5c261;">'name'</span>: <span style="color: #a5c261;">'Bug'</span>}<span style="color: #cc7832;">,</span>
<span style="color: #cc7832;"> </span><span style="color: #a5c261;">'labels'</span>: [label]
}
<span style="color: #cc7832; font-weight: bold;">return </span>JIRARestAPI.jira.create_issue(<span style="color: #aa4926;">fields</span>=issue_dict)</pre>
</div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-24725220273339005552013-04-09T01:05:00.001-07:002013-04-09T01:05:09.442-07:00Per thread CPU usage on Linux<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"></b><br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">To investigate the per-thread CPU usage on Linux, use command ‘top’ with the -H option, which provides an additional per thread information, which is not provided by default ‘top’ usage.</span></b></div>
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif;"><br /><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></span></b>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">The output of ‘top -H’ on Linux shows the breakdown of the CPU usage on the machine by individual threads. The top output has the following sections of interest:</span></b></div>
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;">
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span>
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: 11px; vertical-align: baseline; white-space: pre-wrap;"> top - 16:15:45 up 21 days, 2:27, 3 users, load average: 17.94, 12.30, 5.52 </span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: #cccccc; font-size: 11px; vertical-align: baseline; white-space: pre-wrap;"> Tasks: 150 total, 26 running, 124 sleeping, 0 stopped, 0 zombie </span><span style="background-color: #cccccc; font-size: 11px; vertical-align: baseline; white-space: pre-wrap;"> Cpu(s): 87.3% us, 1.2% sy, 0.0% ni, 27.6% id, 0.0% wa, 0.0% hi, 0.0% si </span><span style="background-color: #cccccc; font-size: 11px; vertical-align: baseline; white-space: pre-wrap;"> Mem: 4039848k total, 3999776k used, 40072k free, 92824k buffers </span><span style="background-color: #cccccc; font-size: 11px; vertical-align: baseline; white-space: pre-wrap;"> Swap: 2097144k total, 224k used, 2096920k free, 1131652k cached </span></span><br />
<br />
</div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">The Cpus(s) row in this header section shows the CPU usage in terms of the following:</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">us - Percentage of CPU time spent in user space.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">sy - Percentage of CPU time spent in kernel space.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">ni - Percentage of CPU time spent on low priority processes.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">id - Percentage of CPU time spent idle.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">wa - Percentage of CPU time spent in wait (on disk).</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">hi - Percentage of CPU time spent handling hardware interrupts.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">si - Percentage of CPU time spent handling software interrupts.</span></div>
<span style="font-family: Times, Times New Roman, serif;"><br /><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">The "us", "sy" and "id" values are useful as the user, system (kernel) and idle CPU time respectively.</span></div>
<span style="font-family: Times, Times New Roman, serif;"><br /><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">The next section shows the per-thread breakdown of the CPU usage.</span><br />
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="vertical-align: baseline; white-space: pre-wrap;"><span style="background-color: #cccccc;"><span style="font-family: Arial;"><span style="font-size: 15px;"> </span></span><span style="font-family: Courier New;"><span style="font-size: 11px;">PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br class="kix-line-break" /> 31253 user1 16 0 2112m 2.1g 1764 R 37.0 53.2 0:39.89 java <br class="kix-line-break" /> 31249 user1 16 0 2112m 2.1g 1764 R 15.5 53.2 0:38.29 java <br class="kix-line-break" /> 31244 user1 16 0 2112m 2.1g 1764 R 13.6 53.2 0:40.05 java <br class="kix-line-break" /> 31250 user1 16 0 2112m 2.1g 1764 R 13.6 53.2 0:41.23 java <br class="kix-line-break" /> 31242 user1 16 0 2112m 2.1g 1764 R 12.9 53.2 0:40.56 java <br class="kix-line-break" /> 31238 user1 16 0 2112m 2.1g 1764 S 12.6 53.2 1:22.21 java <br class="kix-line-break" /> 31246 user1 16 0 2112m 2.1g 1764 R 12.6 53.2 0:39.62 java <br class="kix-line-break" /> 31248 user1 16 0 2112m 2.1g 1764 R 12.6 53.2 0:39.40 java <br class="kix-line-break" /> 31258 user1 16 0 2112m 2.1g 1764 R 12.6 53.2 0:39.98 java <br class="kix-line-break" /> 31264 user1 17 0 2112m 2.1g 1764 R 12.6 53.2 0:39.54 java <br class="kix-line-break" /> 31243 user1 16 0 2112m 2.1g 1764 R 12.2 53.2 0:37.43 java <br class="kix-line-break" /> 31245 user1 16 0 2112m 2.1g 1764 R 12.2 53.2 0:37.53 java </span></span></span><span style="font-family: Courier New;"><span style="background-color: #cccccc; font-size: 11px;"> </span><span style="background-color: #cccccc; font-size: 11px;"> ...</span></span></span><br />
<span style="vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Courier New;"><span style="font-size: 11px;"><br class="kix-line-break" /></span></span><span style="font-size: 15px;"><span style="font-family: Times, Times New Roman, serif;">This provides the following information per thread basis:</span></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif;"><b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"></b></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">PID - The thread ID. This can be converted into hexadecimal and used to correlate to the "native ID" in a java thread dump file.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">USER - The user ID of the user that started the process.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">PR - The priority of the thread.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">NI - The "nice" value for the process.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">VIRT - The virtual memory (allocated) usage of the process.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">RES - The resident memory (committed) usage of the process.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">SHR - The shared memory usage of the process.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">S - The state of the thread. This can be one of the following:</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">R - Running</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">S - Sleeping</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">D - Uninterruptible sleep</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">T - Traced</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 72pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Z - Zombie</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; font-weight: bold; vertical-align: baseline; white-space: pre-wrap;">%CPU - The percentage of a single CPU usage by the thread.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">%MEM - The percentage of the memory used by the process.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">TIME+ - The amount of CPU time used by the thread.</span></b></div>
<span style="font-family: Times, Times New Roman, serif;"><b id="internal-source-marker_0.5426343877334148" style="font-weight: normal;"><div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">COMMAND - The name of the process executable.</span></div>
<div>
<span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
</b><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Note that the "Cpu(s)" line in the header of the output shows the percentage usage across all of the available CPUs, whereas the %CPU column above represents the percentage usage of a single CPU. Thus for example, on a four-CPU machine the Cpu(s) row will total 100% and the %CPU column will total to 400%. To see the per cpu usage in header section, press 1.</span></div>
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<h3>
<span style="color: #073763; font-family: Arial; font-size: 15px; font-weight: bold; vertical-align: baseline; white-space: pre-wrap;">What to look for from the top -H output?</span></h3>
</div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">In the per-thread breakdown of the CPU usage shown above, the Java process is taking approximately 75% of the CPU usage. This value is found by totaling the %CPU column for all the Java threads (not all threads are shown above) and dividing by the number of CPUs. The Java process is not limited by other processes, because the CPU there is still approximately 25% idle. </span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">You can also see that the CPU usage of the Java process is spread reasonably evenly over all of the threads in the Java process. This spread implies that no one thread has a particular problem. Although the application is allowed to use most of the available CPU, approximately 25% of the total CPU is idle meaning that some points of contention or delay in the Java process can be identified.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">A report indicating that active processes are using a small percentage of CPU, even though the machine appears idle, means that the performance of the application is probably limited by points of contention or process delay, preventing the application from scaling to use all of the available CPU. </span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">If a deadlock is present, the reported CPU usage for the Java process is low or zero. </span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">If threads are looping, the Java CPU usage approaches 100%, but a small number of the threads account for all of that CPU time. </span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Whenever you have threads of interest, note the PID values, convert them to a hexadecimal value and look up the threads in thread dump file to discover the name of application thread. Then look at the thread stack trace to understand the kind of work it is doing. </span></div>
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<h3>
<span style="color: #073763; font-family: Arial; font-size: 15px; font-weight: bold; vertical-align: baseline; white-space: pre-wrap;">How to generate thread dump for Java process?</span></h3>
</div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Times, Times New Roman, serif; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">The thread dump for a java process can be generated using command:</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: #cccccc; font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">/bin/kill -3 <pid></span></div>
<div>
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
</b>Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com2tag:blogger.com,1999:blog-9166453787943931153.post-31367015125826082072013-03-27T06:09:00.000-07:002013-10-29T04:31:33.712-07:00“Too many open files” error in highly multithreaded application<div dir="ltr" style="text-align: left;" trbidi="on">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"></b><br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<u><b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Error:</span></b></u></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif;">Caused by: java.io.FileNotFoundException: errors.txt (Too many open files)</span></span></span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>at java.io.FileOutputStream.openAppend(Native Method)</span></span></span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>at java.io.FileOutputStream.<init>(FileOutputStream.java:192)</span></span></span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>at java.io.FileOutputStream.<init>(FileOutputStream.java:116)</span></span></span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>at hydra.FileUtil.appendToFile(FileUtil.java:528)</span></span></span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>... 5 more</span></span></span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">If we see “Too many open files” error in highly multi- threaded application, this means we have reached to maximum number of file handles. </span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b><br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<br />
<u><b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">To see where you are at with file handles on Linux<b>:</b></span></b></u><br />
<br />
<span style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">R</span></span><b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">un the command</span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b><br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Courier New, Courier, monospace;"> sysctl fs.file-nr</span></span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b><br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">From this command you will get 3 numbers. First is the number of used file descriptor the second is the number of allocated but not used file descriptor and the last is the system max file descriptor.</span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
</b><br />
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"></b><br />
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b><u><span style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">To up this limit:</span></span></u></b><br />
<span style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Y</span></span><b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">ou will need to add something similar to the following in the file <b>/etc/sysctl.conf</b>:</span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b><br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Courier New, Courier, monospace;"> fs.file-max = 204708</span></span></b></div>
<br />
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"></b><br />
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Also there is an individual users file handle limit, that is where the "<b>ulimit -n</b>" command. If you are running to this limit you will have to add entries to the <b>/etc/security/limits.conf</b> file for the user like the following:</span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
</b>
<br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
</b>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;"><span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Courier New, Courier, monospace;"><b>@users hard nofile 81920<br />@users soft nofile 8192<br />@users hard nproc unlimited<br />@users soft nproc 501408<br />@users soft core unlimited<br />@users hard core unlimited</b></span></span></b></div>
<b id="internal-source-marker_0.9598071791697294" style="font-weight: normal;">
<br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"></span></span></div>
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span><br /><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"></span><div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<u><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Apply the changes using :</span></u></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<br />
<span style="font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Courier New, Courier, monospace;"> sysctl -a</span></span></div>
<div>
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
</b></div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0tag:blogger.com,1999:blog-9166453787943931153.post-47233928579861132312013-02-27T00:33:00.003-08:002013-02-27T00:33:27.284-08:00OutOfMemoryError because of default stack size<span id="internal-source-marker_0.46449039418205185" style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;">Error String: </span><br />
<blockquote class="tr_bq">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"> java.lang.OutOfMemoryError: unable to create new native thread<br class="kix-line-break" /> at java.lang.Thread.start0(Native Method)<br class="kix-line-break" /> at java.lang.Thread.start(Thread.java:597)</span></blockquote>
<br />
<span id="internal-source-marker_0.46449039418205185" style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"><span id="internal-source-marker_0.2735328081935203" style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;">The
application dies with “java.lang.OutOfMemoryError: unable to create new
native thread.” The solution is to reduce the stack size. The JVM has
an interesting implementation, the design of which I don’t completely
understand, but the implication is that the more memory is allocated for
the heap (not necessarily used by the heap), the less memory available
in the stack, and since threads are made from the stack, in practice
this means more “memory” in the heap sense (which is usually what people
talk about) results in less threads being able to run concurrently.</span><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"></span><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;">Each
thread by default gets a stack of 1MB. In a highly multithreaded system
(~1500 threads), it is possible that we run out of contiguous blocks of
memory on that machine. The limitation is normally stack space (which
must be in contiguous blocks) and since every thread consumes this
scattered about you rapidly run out of contiguous blocks. Reference:<a href="http://stackoverflow.com/questions/481900/whats-the-maximum-number-of-threads-in-windows-server-2003">http://stackoverflow.com/questions/481900/whats-the-maximum-number-of-threads-in-windows-server-2003</a></span><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"></span><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;">To
reduce the stack size, add “-Xss64kb” to the JVM options. Start with
64k, try the application, then if it doesn’t work (it will fail with a
Java.lang.StackOverFlowError), increase the stack to 128k, then 256k,
and so on. The default stack size is 8192k so there’s a wide range to
test.</span><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;">Also,
on Linux, you’ll need to set the Linux thread stack size to the same
value as the JVM stack size to get full benefits. To do that, use
“ulimit -s <size in kb>”. Note that the stack size applies per
user, so you have to modify the init script, or edit
/etc/security/limits.conf (on Debian/Ubuntu at least).</span><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"> </span></span><br />
<span id="internal-source-marker_0.46449039418205185" style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline;"><a href="http://www.vmware.com/in/products/application-platform/vfabric-gemfire/overview.html">GemFire</a>
is a highly multi-threaded system and at any given point in time there
are multiple thread pools and threads that are in use. The default stack
size setting for a thread in java is 1MB. Stack size has to be
allocated in contiguous blocks and if the machine is being used actively
and there are many threads running in the system (Task Manager shows
the number of active threads), you may encounter <i>OutOfMemoryError:
unable to create new native thread</i> , even though your process has enough
available heap. If this happens, consider reducing the stack size
requirement for threads on the cache server. The stack size setting for
cache servers can be set using the -Xss vm arg. This can be set to 384k
or 512k in such cases </span><br /> </span><br />
<br />
<br />
<br />
<br />
<div dir="ltr" style="margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<br /></div>
Anonymoushttp://www.blogger.com/profile/15751358960711079615noreply@blogger.com0