{"id":1163,"date":"2015-05-16T10:00:52","date_gmt":"2015-05-16T18:00:52","guid":{"rendered":"http:\/\/www.developerscloset.com\/?page_id=1163"},"modified":"2018-05-16T10:02:36","modified_gmt":"2018-05-16T18:02:36","slug":"hue","status":"publish","type":"page","link":"https:\/\/www.developerscloset.com\/?page_id=1163","title":{"rendered":"Hue"},"content":{"rendered":"<p><a href=\"http:\/\/www.developerscloset.com\/wp-content\/uploads\/2018\/05\/hue.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-1164 alignnone\" src=\"http:\/\/www.developerscloset.com\/wp-content\/uploads\/2018\/05\/hue.png\" alt=\"\" width=\"248\" height=\"62\" \/><\/a><\/p>\n<p>Hue is a graphical user interface for Hadoop. Hue applications are collected into a desktop-style environment and delivered as a Web application, requiring no additional installation for individual users.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_79 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-69ea242197c7c\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-69ea242197c7c\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Configure_Hue\" >Configure Hue<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Install_Hue\" >Install Hue<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Install_and_Configure_Hue\" >Install and Configure Hue<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Configure_an_LDAP_Backend\" >Configure an LDAP Backend<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Test_Hue\" >Test Hue<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Inspecting_the_Hue_Database\" >Inspecting the Hue Database<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Troubleshooting\" >Troubleshooting<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_Cannot_See_Pig_Scripts_After_Upgrade\" >Hue Cannot See Pig Scripts After Upgrade<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_is_Running_Slow\" >Hue is Running Slow<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_is_not_responding_%E2%80%93_DatabaseError_database_is_locked\" >Hue is not responding &#8211; DatabaseError: database is locked<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_Cannot_access_Spark_from_Hue\" >Hue: Cannot access Spark from Hue<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_Cannot_run_Pig_Scripts_from_Hue_to_YARN_using_MRv2\" >Hue: Cannot run Pig Scripts from Hue to YARN (using MRv2)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_Cannot_Open_Workflow_Editor\" >Hue: Cannot Open Workflow Editor<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/www.developerscloset.com\/?page_id=1163\/#Hue_Cannot_Create_a_New_Workflow\" >Hue: Cannot Create a New Workflow<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1 id=\"Hue-ConfigureHue\"><span class=\"ez-toc-section\" id=\"Configure_Hue\"><\/span>Configure Hue<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<h2 id=\"Hue-InstallHue\"><span class=\"ez-toc-section\" id=\"Install_Hue\"><\/span>Install Hue<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Cloudera Manager distributes Hue in CDH and offers the following services:<\/p>\n<ul>\n<li><strong>Hue Server<\/strong>\u00a0&#8211; For small clusters of less than 10 nodes, you can place the Hue service on the same node as the active HDFS NameNode. For larger clusters or for production expect Hue to require more memory &#8211; and configure Hue to use MySQL instead of the default\u00a0PostgreSQL database. To use Hue with HBase, make sure that the HBase Thrift service is installed (see HBase for more information about the HBase Thrift service).<\/li>\n<\/ul>\n<h2 id=\"Hue-InstallandConfigureHue\"><span class=\"ez-toc-section\" id=\"Install_and_Configure_Hue\"><\/span>Install and Configure Hue<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<ol>\n<li>Browse to Cloudera Manager, select the\u00a0arrow down next to the\u00a0&#8220;host&#8221;<\/li>\n<li>Select\u00a0&#8220;add service&#8221;<\/li>\n<li>Select\u00a0Hue<\/li>\n<li>On &#8220;Add Service Wizard&#8221; page click on the box under Hue Service<\/li>\n<li>Select Node running the\u00a0<u>Active<\/u>\u00a0Name Node (NN)<\/li>\n<li>Click continue<\/li>\n<li>The service will then install and restart<\/li>\n<li>Deploy client configuration and restart (likely will require a restart)<\/li>\n<li>To configure the service click on Hue from Cloudera Manager site<\/li>\n<li>Click configuration<\/li>\n<li>Search for each configuration below from the search box located under &#8220;filters&#8221;<\/li>\n<\/ol>\n<h2><span class=\"ez-toc-section\" id=\"Configure_an_LDAP_Backend\"><\/span>Configure an LDAP Backend<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>On the main Cloudera Manager site, click on Hue, and select Configurations. Click on the Security category.<\/p>\n<div style=\"max-width: 100%;margin: auto;overflow: hidden\">\n<div style=\"width: 100%;overflow: auto\">\n<table class=\"wrapped confluenceTable\">\n<colgroup>\n<col \/>\n<col \/><\/colgroup>\n<tbody>\n<tr>\n<td class=\"confluenceTd\"><strong>Configuration<\/strong><\/td>\n<td class=\"confluenceTd\"><strong>Value<\/strong><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>Authentication Backend<\/strong><\/p>\n<p>backend<\/td>\n<td class=\"confluenceTd\"><u>desktop.auth.backend.LdapBackend<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP URL<\/strong><\/p>\n<p>ldap_url<\/td>\n<td class=\"confluenceTd\"><u><a rel=\"nofollow\">ldap:\/\/company.com<\/a><\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\" colspan=\"1\">\n<p class=\"display-name\"><strong>Enable LDAP TLS<\/strong><\/p>\n<p class=\"display-name\">use_start_tls<\/p>\n<\/td>\n<td class=\"confluenceTd\" colspan=\"1\">True<\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>Active Directory Domain<\/strong><\/p>\n<p>nt_domain<\/td>\n<td class=\"confluenceTd\"><u><a rel=\"nofollow\">company.com<\/a><\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>Create LDAP users on login<\/strong><\/p>\n<p>create_users_on_login<\/td>\n<td class=\"confluenceTd\">True<\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP Search Base<\/strong><\/p>\n<p>base_dn<\/td>\n<td class=\"confluenceTd\"><u>OU=Organization,DC=company,DC=com<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP Bind User Distinguished Name<\/strong><\/p>\n<p>bind_dn<\/td>\n<td class=\"confluenceTd\"><u>HUEServerName##<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP Bind Password<\/strong><\/p>\n<p>bind_password<\/td>\n<td class=\"confluenceTd\"><u>*****<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP User Filter<\/strong><\/p>\n<p>user_filter<\/td>\n<td class=\"confluenceTd\"><u>objectclass=*<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP Username Attribute<\/strong><\/p>\n<p>user_name_attr<\/td>\n<td class=\"confluenceTd\"><u>sAMAccountName<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP Group Filter<\/strong><\/p>\n<p>group_filter<\/td>\n<td class=\"confluenceTd\"><u>objectclass=*<\/u><\/td>\n<\/tr>\n<tr>\n<td class=\"confluenceTd\"><strong>LDAP Group Name Attribute<\/strong><\/p>\n<p>group_name_attr<\/td>\n<td class=\"confluenceTd\"><u>cn<\/u><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<p>For more information, refer to the Hue Installation Guide:\u00a0<a class=\"external-link\" href=\"http:\/\/cloudera.github.io\/hue\/docs-2.0.1\/manual.html\" rel=\"nofollow\">http:\/\/cloudera.github.io\/hue\/docs-2.0.1\/manual.html<\/a><\/p>\n<h1><span class=\"ez-toc-section\" id=\"Test_Hue\"><\/span>Test Hue<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<ol>\n<li>connect to\u00a0<a rel=\"nofollow\">http:\/\/hue.servername01:9090<\/a><\/li>\n<li>Log in with your Hue credentials<\/li>\n<\/ol>\n<p>If you fail to connect to the Hue UI there may be a problem with The HBase Thrift server:\u00a0After you install Hue, you need to make sure that your HBase installation has the HBase Thrift Server installed or you will receive this error from the Hue HBase browser: HBase browser couldn&#8217;t connect to localhost:9090<\/p>\n<p>Here is the reason why: In Hue 2.5.0, there is a new feature called &#8220;HBase Browser&#8221;, it is for user to quickly browsing huge tables and accessing HBase content. You can also create new tables, add data, modify existing cells and filter data with the auto-completing search bar. If you click on &#8220;HBase Browser&#8221; icon and get &#8220;API error: couldn&#8217;t connect to localhost:9090&#8221;, probably you don&#8217;t have a HBase thrift server running.<\/p>\n<p>And how to fix this: In your CM, go to &#8220;All Services&#8221; -&gt; &#8220;hbase1&#8221; -&gt; &#8220;Instances&#8221;, then under &#8220;Role Instances&#8221;, click on &#8220;Add&#8221;, choose a node to be &#8220;HBase Thrift Server&#8221;, then start the Thrift server. By default, Hue connects to itself on port 9090, so make sure Hue knows which node is the Thrift server.<\/p>\n<h1 id=\"Hue-InspectingtheHueDatabase\"><span class=\"ez-toc-section\" id=\"Inspecting_the_Hue_Database\"><\/span>Inspecting the Hue Database<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<p>Hue requires an SQL database to store small amounts of data, including user account information as well as history of job submissions and Hive queries. By default, Hue is configured to use either PostgreSQL or an embedded database SQLite for this purpose, and should require no configuration or management by the administrator. However, MySQL is the recommended database to use; this section contains instructions for configuring Hue to access MySQL and other databases.<\/p>\n<p>The default SQLite database used by Hue is located in \/usr\/share\/hue\/desktop\/desktop.db. You can inspect this database from the command line using the sqlite3 program.<\/p>\n<p>Pig Scripts are located in the following tables:<\/p>\n<blockquote><p>pig_document<\/p>\n<p>pig_pigscript<\/p><\/blockquote>\n<p>For example:<\/p>\n<blockquote><p># sqlite3 \/var\/lib\/hue\/desktop.db<\/p>\n<p>SQLite version 3.6.22<\/p>\n<p>Enter &#8220;.help&#8221; for instructions<\/p>\n<p>Enter SQL statements terminated with a &#8220;;&#8221;<\/p>\n<p>sqlite&gt; .tables<\/p>\n<p>sqlite&gt; .schema auth_user<\/p>\n<p>sqlite&gt; select username from auth_user;<\/p>\n<p>admin<\/p>\n<p>test<\/p>\n<p>sample<\/p>\n<p>sqlite&gt; .quit<\/p><\/blockquote>\n<h1 id=\"Hue-Troubleshooting\"><span class=\"ez-toc-section\" id=\"Troubleshooting\"><\/span>Troubleshooting<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<h2 id=\"Hue-HueCannotSeePigScriptsAfterUpgrade\"><span class=\"ez-toc-section\" id=\"Hue_Cannot_See_Pig_Scripts_After_Upgrade\"><\/span>Hue Cannot See Pig Scripts After Upgrade<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Missing Pig scripts: After upgrading from CDH4.7 to CDH5.1.0 the Hue landing page displays this error:<\/p>\n<p>Server Error (500)<\/p>\n<p>Sorry, there&#8217;s been an error. An email was sent to your administrators. Thank you for your patience.<\/p>\n<p><strong>More Info:<\/strong><\/p>\n<p>File Name Line Number Function Name<\/p>\n<p>\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.6\/site-packages\/Django-1.4.5-py2.6.egg\/django\/core\/handlers\/base.py 111 get_response<\/p>\n<p>\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/desktop\/core\/src\/desktop\/views.py 56 home<\/p>\n<p>\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/desktop\/core\/src\/desktop\/api.py 37 _get_docs<\/p>\n<p>&#8230;<\/p>\n<p>\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.6\/site-packages\/Django-1.4.5-py2.6.egg\/django\/db\/models\/sql\/compiler.py763 results_iter<\/p>\n<p>\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.6\/site-packages\/Django-1.4.5-py2.6.egg\/django\/db\/models\/sql\/compiler.py818 execute_sql<\/p>\n<p>\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.6\/site-packages\/Django-1.4.5-py2.6.egg\/django\/db\/backends\/sqlite3\/base.py 344 execute<\/p>\n<p><strong>Cause:<\/strong>\u00a0Upgrade failed to create all the required tables for Hue.<\/p>\n<p><strong>Resolution:<\/strong><\/p>\n<p>1.\u00a0Go to the Hue directory:<\/p>\n<div>\n<blockquote><p>cd \/var\/lib\/hue<\/p><\/blockquote>\n<\/div>\n<p>2. Backup the database:<\/p>\n<div>\n<blockquote><p>cp desktop.db desktop.db.back<\/p><\/blockquote>\n<\/div>\n<p>3. Sync the database by running syncdb:<\/p>\n<div>\n<blockquote><p>\/opt\/cloudera\/parcels\/CDH\/lib\/hue\/build\/env\/bin\/hue syncdb &#8211;noinput<\/p><\/blockquote>\n<\/div>\n<p>4. Run the following:<\/p>\n<div>\n<blockquote><p>\/opt\/cloudera\/parcels\/CDH\/lib\/hue\/build\/env\/bin\/hue migrate &#8211;delete-ghost-migrations<\/p><\/blockquote>\n<\/div>\n<h2 id=\"Hue-HueisRunningSlow\"><span class=\"ez-toc-section\" id=\"Hue_is_Running_Slow\"><\/span>Hue is Running Slow<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>On some installations the Hue service shares its node with the Cloudera Manager Service, which can use quite a bit of memory. If Hue is running slow it is possible that the node is too busy. Restart the Cloudera Manager Service and watch memory. Consider reinstalling Hue on another node.<\/p>\n<ol>\n<li>Check if memory is a problem on the node, browse to Cloudera Manager, select the node.<\/li>\n<li>How much memory is used? Is it in the red zone, or yellow? For example, 80% used is generally good for Hue. Too much higher and you will notice slowness.<\/li>\n<li>If too much memory is in use, restart the Cloudera Manager Service.<\/li>\n<li>In Cloudera Manager, click Clusters, and select Cloudera Manager Service.<\/li>\n<li>Within the Cloudera Manager Service, click Actions, Restart.<\/li>\n<li>Make sure the service comes back up. You should notice that the memory used has gone down quite a bit and Hue is a little more responsive.<\/li>\n<\/ol>\n<h2 id=\"Hue-Hueisnotresponding-DatabaseError:databaseislocked\"><span class=\"ez-toc-section\" id=\"Hue_is_not_responding_%E2%80%93_DatabaseError_database_is_locked\"><\/span>Hue is not responding &#8211; DatabaseError: database is locked<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><strong>Problem:\u00a0<\/strong>Hue does not open, the website spins but does not present a page.<\/p>\n<p><strong>Resolution:<\/strong>\u00a0I had to restart the service twice, on the second time I took Hue completely down for about a minute to make sure the database had stopped completely. I then started the service and Hue was able to connect.<\/p>\n<p>In the log: I see the following:<\/p>\n<p>DatabaseError: database is locked<\/p>\n<p>[14\/Oct\/2014 13:10:00 -0700] base\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ERROR\u00a0\u00a0\u00a0 Internal Server Error: \/pig\/dashboard\/<\/p>\n<p>Traceback (most recent call last):<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/core\/handlers\/base.py&#8221;, line 111, in get_response<\/p>\n<p>response = callback(request, *callback_args, **callback_kwargs)<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/apps\/oozie\/src\/oozie\/views\/dashboard.py&#8221;, line 88, in decorate<\/p>\n<p>return view_func(request, *args, **kwargs)<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/apps\/pig\/src\/pig\/views.py&#8221;, line 58, in dashboard<\/p>\n<p>hue_jobs = Document.objects.available(PigScript, request.user, with_history=True)<\/p>\n<p>&nbsp;<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/db\/models\/query.py&#8221;, line 445, in get_or_create<\/p>\n<p>return self.get(**lookup), False<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/db\/models\/sql\/compiler.py&#8221;, line 818, in execute_sql<\/p>\n<p>cursor.execute(sql, params)<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.1.0-1.cdh5.1.0.p0.53\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/db\/backends\/sqlite3\/base.py&#8221;, line 344, in execute<\/p>\n<p>return Database.Cursor.execute(self, query, params)<\/p>\n<p>DatabaseError: database is locked<\/p>\n<p>[14\/Oct\/2014 15:00:51 -0700] api\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ERROR\u00a0\u00a0 \u00a0An error happen while watching the demo running: &#8216;NoneType&#8217; object has no attribute &#8216;group&#8217;<\/p>\n<p>[14\/Oct\/2014 15:00:51 -0700] api\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ERROR\u00a0\u00a0\u00a0 An error happen while watching the demo running: &#8216;NoneType&#8217; object has no attribute &#8216;group&#8217;<\/p>\n<p>[14\/Oct\/2014 15:00:52 -0700] api\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ERROR\u00a0\u00a0\u00a0 An error happen while watching the demo running: &#8216;NoneType&#8217; object has no attribute &#8216;group&#8217;<\/p>\n<h2 id=\"Hue-Hue:CannotaccessSparkfromHue\"><span class=\"ez-toc-section\" id=\"Hue_Cannot_access_Spark_from_Hue\"><\/span>Hue: Cannot access Spark from Hue<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>An error happened with the Spark Server:<\/p>\n<p>HTTPConnectionPool(host=&#8217;localhost&#8217;, port=8090): Max retries exceeded with url: \/jobs (Caused by &lt;class &#8216;socket.error&#8217;&gt;: [Errno 111] Connection refused)<\/p>\n<p>Under Hue Configuration (within Cloudera Manager) \/ Advanced \/ Hue Server Advanced Configuration Snippet (Safety Valve) for hue_safety_valve_server.ini<\/p>\n<p>Add the following section:<\/p>\n<div>\n<blockquote><p>[spark]<\/p>\n<p># URL of the REST Spark Job Server.<\/p>\n<p>server_url=http:\/\/spark.rest.servername01:18080\/<\/p><\/blockquote>\n<\/div>\n<p>See the Configure Spark section for more information.<\/p>\n<h2 id=\"Hue-Hue:CannotrunPigScriptsfromHuetoYARN(usingMRv2)\"><span class=\"ez-toc-section\" id=\"Hue_Cannot_run_Pig_Scripts_from_Hue_to_YARN_using_MRv2\"><\/span>Hue: Cannot run Pig Scripts from Hue to YARN (using MRv2)<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><strong>Resolution:<\/strong>\u00a0YARN\u2019s resources were set too low (memory was set to 50 MB, when it should have been set to 1 GB).<\/p>\n<p>I tried to narrow down the problem I&#8217;m having with running Pig scripts through Hue and YARN. Here is what I do:<\/p>\n<p>1.\u00a0Create a Pig Script in Hue:<\/p>\n<div>\n<blockquote><p>offers = LOAD &#8216;\/tmp\/datafile.txt&#8217; USING PigStorage AS (name:CHARARRAY);<\/p><\/blockquote>\n<\/div>\n<p>The script succeeds.<\/p>\n<p>2. However, when I add a dump to the script, like this:<\/p>\n<div>\n<blockquote><p>offers = LOAD &#8216;\/tmp\/datafile.txt&#8217; USING PigStorage AS (name:CHARARRAY);<\/p>\n<p>dump offers;<\/p><\/blockquote>\n<\/div>\n<p>To see the log, click on the status of the Pig job in the top right corner, it will open its Oozie workflow, then click on the Pig action on the log icon on the right. You should have more interesting logs!<\/p>\n<p>For example, in the log I see this line: 2014-08-14 16:24:35,692 [main] INFO\u00a0 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher\u00a0 &#8211; More information at:\u00a0http:\/\/node.servername05:50030\/jobdetails.jsp?jobid=job_1408018429315_0002<\/p>\n<p>The script never moves past 0% and repeats Heat beat over and over again. The job displays in Oozie but never goes anywhere (the job is stuck on RUNNING). This same script worked in CDH 4.7 using MRv1. I can&#8217;t find much in the logs to help identify a problem, it just never finishes.<\/p>\n<p>Here is an excerpt from the job&#8217;s log:<\/p>\n<p>2014-08-19 14:31:01,128 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher &#8211; More information at:\u00a0http:\/\/node.servername05:50030\/jobdetails.jsp?jobid=job_1408403413938_0014<\/p>\n<p>2014-08-19 14:31:01,227 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher &#8211; 0% complete<\/p>\n<p>Heart beat<\/p>\n<p>&#8230;<\/p>\n<p>&nbsp;<\/p>\n<p>Did not work:<\/p>\n<ul>\n<li>Reinstall the Oozie sharelib, 1. Stop Oozie, 2. Under Actions, select Install Sharelib 3. Make sure that the sharelib is using the one for Yarn: oozie-sharelib-yarn.tar.gz<\/li>\n<li>Click on &#8216;Hue server&#8217;, stop it, then do &#8216;Synchronize Database&#8217; and restart Hue.<\/li>\n<li>I applied the change from step #5 in the document:\u00a0<a class=\"external-link\" href=\"http:\/\/blog.cloudera.com\/blog\/2014\/04\/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas\/\" rel=\"nofollow\">http:\/\/blog.cloudera.com\/blog\/2014\/04\/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas\/<\/a>, but unfortunately, it did not help. But this looks very similar to my problem.<\/li>\n<\/ul>\n<p>For information on how to configure Yarn, see Configure Yarn, specifically, Configure Yarn Resources.<\/p>\n<h2 id=\"Hue-Hue:CannotOpenWorkflowEditor\"><span class=\"ez-toc-section\" id=\"Hue_Cannot_Open_Workflow_Editor\"><\/span>Hue: Cannot Open Workflow Editor<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><strong>Problem:<\/strong>\u00a0Hue cannot edit an Oozie workflow.<\/p>\n<p>Open Hue, click Workflow, and select Editor.<\/p>\n<p>Receive the error: Server Error (500)<\/p>\n<p><strong>Resolution:<\/strong>\u00a0After some debugging (see below), on the node that is running Hue can you run this from a bash shell:<\/p>\n<div>\n<p>\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/build\/env\/bin\/hue migrate &#8211;delete-ghost-migrations<\/p>\n<\/div>\n<p>Results from the migrate command:<\/p>\n<p>Running migrations for desktop:<\/p>\n<p>&#8211; Migrating forwards to 0007_auto__add_documentpermission__add_documenttag__add_document.<\/p>\n<p>&gt; desktop:0007_auto__add_documentpermission__add_documenttag__add_document<\/p>\n<p>&#8211; Loading initial data for desktop.<\/p>\n<p>Installed 0 object(s) from 0 fixture(s)<\/p>\n<p>&nbsp;<\/p>\n<p>&#8230;<\/p>\n<p>Running migrations for oozie:<\/p>\n<p>&#8211; Migrating forwards to 0025_change_examples_path_format.<\/p>\n<p>&gt; oozie:0022_auto__chg_field_mapreduce_node_ptr__chg_field_start_node_ptr<\/p>\n<p>&gt; oozie:0023_auto__add_field_node_data__add_field_job_data<\/p>\n<p>&gt; oozie:0024_auto__chg_field_subworkflow_sub_workflow<\/p>\n<p>&gt; oozie:0025_change_examples_path_format<\/p>\n<p>&#8211; Migration &#8216;oozie:0025_change_examples_path_format&#8217; is marked for no-dry-run.<\/p>\n<p>&#8211; Loading initial data for oozie.<\/p>\n<p>Installed 0 object(s) from 0 fixture(s)<\/p>\n<p>&nbsp;<\/p>\n<p>&#8230;<\/p>\n<p>south.exceptions.GhostMigrations:<\/p>\n<p>! These migrations are in the database but not on disk:<\/p>\n<p>&lt;oozie: 0022_change_examples_path_format&gt;<\/p>\n<p>! I&#8217;m not trusting myself; either fix this yourself by fiddling<\/p>\n<p>! with the south_migrationhistory table, or pass &#8211;delete-ghost-migrations<\/p>\n<p>! to South to have it delete ALL of these records (this may not be good).<\/p>\n<p>The error points to a problem in Oozie:<\/p>\n<p>[30\/Jun\/2014 09:14:17 -0700] base\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ERROR\u00a0\u00a0\u00a0 Internal Server Error: \/oozie\/list_workflows\/<\/p>\n<p>Traceback (most recent call last):<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/core\/handlers\/base.py&#8221;, line 111, in get_response<\/p>\n<p>response = callback(request, *callback_args, **callback_kwargs)<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/apps\/oozie\/src\/oozie\/views\/editor.py&#8221;, line 64, in list_workflows<\/p>\n<p>data = Document.objects.available(Workflow, request.user)<\/p>\n<p>&#8230;<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/db\/models\/sql\/compiler.py&#8221;, line 818, in execute_sql<\/p>\n<p>cursor.execute(sql, params)<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/db\/backends\/sqlite3\/base.py&#8221;, line 344, in execute<\/p>\n<p>return Database.Cursor.execute(self, query, params)<\/p>\n<p>DatabaseError: no such table: desktop_documenttag<\/p>\n<p>[30\/Jun\/2014 09:14:17 -0700] middleware\u00a0\u00a0 INFO\u00a0\u00a0\u00a0\u00a0 Processing exception: no such table: desktop_documenttag: Traceback (most recent call last):<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/core\/handlers\/base.py&#8221;, line 111, in get_response<\/p>\n<p>response = callback(request, *callback_args, **callback_kwargs)<\/p>\n<p>&#8230;<\/p>\n<p>File &#8220;\/opt\/cloudera\/parcels\/CDH-5.0.2-1.cdh5.0.2.p0.13\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.4.5-py2.7.egg\/django\/db\/backends\/sqlite3\/base.py&#8221;, line 344, in execute<\/p>\n<p>return Database.Cursor.execute(self, query, params)<\/p>\n<p>DatabaseError: no such table: desktop_documenttag<\/p>\n<p>[30\/Jun\/2014 09:14:16 -0700] access\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 INFO\u00a0\u00a0\u00a0\u00a0 192.168.200.157 admin &#8211; &#8220;GET \/oozie\/list_workflows\/ HTTP\/1.1&#8221;<\/p>\n<h2 id=\"Hue-Hue:CannotCreateaNewWorkflow\"><span class=\"ez-toc-section\" id=\"Hue_Cannot_Create_a_New_Workflow\"><\/span>Hue: Cannot Create a New Workflow<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>User receives a 500\u00a0Server error when they click on the Workflow Editor and attempt to Create a new Workflow.<\/p>\n<p><strong>Error:\u00a0<\/strong>User: httpfs is not allowed to impersonate hue (error 500)<\/p>\n<p>On Hue&#8217;s web UI we see the following: 500\u00a0Server error:\u00a0Sorry, there&#8217;s been an error. An email was sent to your administrators. Thank you for your patience.<\/p>\n<p>Within Hue&#8217;s log file we see:<\/p>\n<p>sudo less \/var\/log\/hue\/runcpserver.log<\/p>\n<p>[12\/Sep\/2017\u00a014:42:58 -0700] connectionpool INFO Resetting dropped connection: servername01<br \/>\n[12\/Sep\/2017\u00a014:42:58 -0700] middleware INFO Processing exception: RemoteException: User: httpfs is not allowed to impersonate hue (error 500): Traceback (most recent call last):<br \/>\nFile\u00a0&#8220;\/opt\/cloudera\/parcels\/CDH-5.7.1-1.cdh5.7.1.p0.11\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.6.10-py2.7.egg\/django\/core\/handlers\/base.py&#8221;, line 112,\u00a0in\u00a0get_response<br \/>\nresponse = wrapped_callback(request, *callback_args, **callback_kwargs)<br \/>\nFile\u00a0&#8220;\/opt\/cloudera\/parcels\/CDH-5.7.1-1.cdh5.7.1.p0.11\/lib\/hue\/build\/env\/lib\/python2.7\/site-packages\/Django-1.6.10-py2.7.egg\/django\/db\/transaction.py&#8221;, line 371,\u00a0in\u00a0inner<br \/>\nreturn\u00a0func(*args, **kwargs)<br \/>\n&#8230;<\/p>\n<p>WebHdfsException: RemoteException: User: httpfs is not allowed to impersonate hue (error 500)<\/p>\n<p class=\"auto-cursor-target\">Narrow down the error within httpfs:<\/p>\n<p class=\"auto-cursor-target\">less \/var\/log\/hadoop-httpfs\/hadoop-cmf-hdfs-HTTPFS-httpfs.servername01.log.out<\/p>\n<div class=\"code panel pdl conf-macro output-block\">\n<div class=\"codeContent panelContent pdl\">\n<div id=\"highlighter_898529\" class=\"syntaxhighlighter sh-confluence nogutter bash\">2017-09-12 15:50:27,723 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hue (auth:PROXY) via httpfs (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: httpfs is not allowed to impersonate hue<\/div>\n<\/div>\n<\/div>\n<p class=\"auto-cursor-target\"><strong>Resolution:<\/strong><\/p>\n<p class=\"auto-cursor-target\">The impersionation account error to HttpFS gave me the clue. We set proxy groups in HDFS to allow us to tighten permissions on this service.\u00a0Permissions, in the form of an impersonation account, were added to protect our HttpFS service from unauthorized read\/writes.<\/p>\n<p class=\"auto-cursor-target\">Find the\u00a0hadoop.proxyuser.httpfs.groups configuration in HDFS and add hue.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hue is a graphical user interface for Hadoop. Hue applications are collected into a desktop-style environment and delivered as a Web application, requiring no additional [&#8230;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"class_list":["post-1163","page","type-page","status-publish","hentry"],"jetpack_shortlink":"https:\/\/wp.me\/P1BQ8S-iL","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages\/1163","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1163"}],"version-history":[{"count":2,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages\/1163\/revisions"}],"predecessor-version":[{"id":1166,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages\/1163\/revisions\/1166"}],"wp:attachment":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1163"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}